Reviewing the Year: AI and Deep Learning in 2017 Unraveled on Denny’s Blog
Introduction:
As we approach the end of the year, it’s time to reflect on the amazing advancements in the field of machine learning that occurred in 2017. One of the biggest success stories was the development of AlphaGo, a reinforcement learning agent that beat the world’s best Go players. AlphaGo Zero took it a step further by learning to play Go from scratch, without any human training data. Another significant milestone was the development of Libratus, a system that defeated top poker players in a heads-up tournament. Evolution strategies also made a comeback in reinforcement learning, with researchers demonstrating their potential in achieving performance comparable to traditional algorithms. In terms of deep learning frameworks, PyTorch and TensorFlow were the stars of 2017. Additionally, there were exciting advancements in the application of deep learning to medical problems, such as the development of algorithms for identifying skin cancer and diagnosing irregular heart rhythms. Overall, 2017 was a year of incredible progress and innovation in the field of machine learning.
Full Article: Reviewing the Year: AI and Deep Learning in 2017 Unraveled on Denny’s Blog
Reinforcement Learning Makes Significant Advances in 2017
In 2017, Reinforcement Learning (RL) made significant strides, with one of the biggest success stories being the AI agent, AlphaGo, developed by DeepMind. AlphaGo, using a combination of human training data and self-play, managed to beat the world’s best Go players. This was a surprise as Go was thought to be out of reach for machine learning algorithms.
AlphaGo Zero, another iteration of the AI, took it a step further by learning to play Go from scratch without any human training data. It utilized a technique published in the Thinking Fast and Slow with Deep Learning and Tree Search paper. Towards the end of the year, AlphaZero emerged, mastering not only Go but also Chess and Shogi using the same techniques. These AI programs even made moves that surprised experienced human players, prompting them to adjust their own play styles. To help players learn from AlphaGo, DeepMind also released AlphaGo Teach.
Reinforcement Learning also made progress in the field of Poker. CMU’s Libratus managed to beat top Poker players in a 20-day tournament, while DeepStack became the first system to defeat professional poker players. Both systems played in a Heads-up setting which is easier than playing at a table with multiple players.
Evolution Algorithms Make a Comeback in Reinforcement Learning
While gradient-based approaches have been successful in supervised learning, Evolution Strategies (ES) have been making a comeback in Reinforcement Learning. ES algorithms can work well in RL because the data is not independent and identically distributed. Additionally, they do not rely on gradients and can scale well to thousands of machines, enabling faster parallel training.
In 2017, OpenAI demonstrated that ES can achieve performance comparable to standard RL algorithms like Deep Q-Learning. Uber’s team further showcased the potential of Genetic Algorithms and novelty search. Their algorithm, using a simple Genetic Algorithm, managed to learn to play difficult Atari Games without any gradient information. This progress indicates that we may see further advancements in 2018.
WaveNets, CNNs, and Attention Mechanisms are on the Rise
Google’s Tacotron 2 text-to-speech system, based on WaveNet, produced impressive audio samples in 2017. WaveNet, an autoregressive model, has also been deployed in Google Assistant and has seen speed improvements. WaveNet has shown promising results in Machine Translation as well.
The trend in Machine Learning is moving away from expensive recurrent architectures that take a long time to train. The Attention is All you Need paper showcased the use of attention mechanisms without recurrence or convolutions to achieve state-of-the-art results at a fraction of the training costs.
2017: The Year of Deep Learning Frameworks
The year 2017 saw a proliferation of Deep Learning frameworks. Facebook’s PyTorch gained popularity due to its dynamic graph construction, which is advantageous for Natural Language Processing tasks. TensorFlow, which released version 1.0 with a stable and backwards-compatible API, also had a successful year. Several companion libraries were launched, such as TensorFlow Fold for dynamic computation graphs and TensorFlow Transform for data input pipelines. Google and Facebook, along with other companies, jumped on the framework bandwagon, introducing their own platforms like Uber’s Pyro and Amazon’s Gluon.
Moreover, to address the increasing number of frameworks, Facebook and Microsoft announced the ONNX open format, which allows sharing of deep learning models across frameworks.
Learning Resources Flourish in 2017
With the growing popularity of Deep Learning and Reinforcement Learning, numerous lectures, bootcamps, and events were recorded and published online in 2017. Various academic conferences, including NIPS, ICLR, and EMNLP, made their conference talks available for public viewing. Additionally, researchers published tutorial and survey papers on arXiv to make cutting-edge research more accessible to the community.
Deep Learning in Healthcare
In healthcare, Deep Learning techniques made headlines by surpassing human experts in certain areas. However, understanding the true breakthroughs requires a medical background. For a comprehensive review, Luke Oakden-Rayner’s series on The End of Human Doctors offers valuable insights. One notable development was Stanford’s Deep Learning algorithm that matched dermatologists’ performance in identifying skin conditions.
As 2017 comes to an end, it is clear that Reinforcement Learning, Deep Learning frameworks, and AI advancements have paved the way for exciting possibilities in the field going forward.
Summary: Reviewing the Year: AI and Deep Learning in 2017 Unraveled on Denny’s Blog
The year 2017 saw significant advancements in the field of Artificial Intelligence, particularly in areas such as Reinforcement Learning, Evolution Algorithms, and Deep Learning frameworks. AlphaGo, a Reinforcement Learning agent, achieved great success by beating world-class Go players. Evolution Strategies also made a comeback, showing comparable performance to traditional Reinforcement Learning algorithms. Deep Learning frameworks such as PyTorch and Tensorflow gained popularity, along with the release of various Reinforcement Learning frameworks. Additionally, there were notable developments in the application of AI to medicine, with Deep Learning algorithms showcasing their potential in diagnosing skin cancer and irregular heart rhythms.
Frequently Asked Questions:
1. What is deep learning and how does it differ from traditional machine learning?
Deep learning is a subset of machine learning that is inspired by the structure and function of the human brain. It involves training artificial neural networks with multiple layers, enabling them to learn abstract representations of data. Unlike traditional machine learning, deep learning algorithms have the ability to automatically extract useful features from raw inputs, without the need for manual feature engineering.
2. How does deep learning work?
Deep learning models consist of multiple layers of interconnected artificial neurons. Data is fed into the input layer, and then propagates through the subsequent layers, with each layer transforming the input based on a set of mathematical functions. The final output layer provides the prediction or classification based on the learned representations. The model undergoes an iterative training process, adjusting the weights of the connections between neurons to minimize the difference between predicted and actual outputs.
3. What are the applications of deep learning?
Deep learning has been successfully applied across various fields. It has revolutionized computer vision, enabling accurate object detection, image and video recognition, and autonomous driving. In natural language processing, deep learning powers language translation, sentiment analysis, and chatbots. It has also shown promise in healthcare for diagnosing diseases from medical images, drug discovery, and genomics research.
4. What are the main challenges of deep learning?
One of the challenges of deep learning is the requirement for large amounts of labeled data to achieve good performance. Deep learning models are usually data-hungry and need substantial quantities of high-quality annotated examples for training. Additionally, deep learning models can be computationally intensive, requiring powerful hardware and significant training time. Overfitting, where the model performs well on training data but poorly on unseen data, is also a challenge that needs to be addressed.
5. What is the future of deep learning?
The future of deep learning looks promising. With continued advancements in hardware technologies, such as specialized GPUs and TPUs, deep learning models can be trained faster and more efficiently. There is ongoing research to address the challenges of interpretability and explainability in deep learning models, making them more transparent and trustworthy. As deep learning continues to evolve, we can expect to see its widespread adoption in various industries, leading to innovative solutions and advancements in artificial intelligence.
