Page 1123

Archive for the ‘Alphazero’ Category

Artificial Intelligence, and the Future of Work Should We Be Worried? – BBN Times

Posted: October 21, 2021 at 1:46 am

without comments

Artificial intelligence is at the top of many lists of the most important skills in today'sjobmarket.

In the last decade or so we have seen a dramatic transition from the AI winter (where AI has not lived up to its hype) to an AI spring (where machines can now outperform humans in a wide range of tasks).

Having spent the last 25 years as an AIresearcher and practitioner, I'm often asked about the implications of this technology on the workforce.

I'm quite often disheartened by the amount of disinformation there is on the internet on this topic, so I've decided to share some of my own thoughts.

The difference between what I am about to write, and what you may have read before elsewhere is due to an inherent bias.Rather than being a pure AI practitioner, my PhD and background is in Cognitive Science - the scientific study of how the mind works, spanning such areas as psychology, neuroscience, philosophy, and artificial intelligence. My area of research has been to look explicitly at how the human mind works, and to reverse engineer these processes in the development of artificial intelligence platforms.Hence, I probably have a better understanding than most of the differences and similarities between human and machine intelligence and how this may play out in the workforce (i.e. what jobs will and will not be replaceable in the future).

So let's begin.

A good place to start this discussion is with the work of Katja Grace and colleagues at the Future of Humanity Institute at the University of Oxford. A few years ago they surveyed the world's leadingAI researchers about when they believed machines would outperform humans on a wide range of tasks. These results are below:

Evidently there were different predictions made as to when different types of work will be able to be performed by machines.But in general, there is consensus that there will be major shifts in the workforce in the next 20 years or so.

In the paper,they define high-level machine intelligence being achieved when unaided machines can accomplish every task better and more cheaply than human workers.Aggregating the data, on average experts believe there is a 50% chance that this will be achieved within 45 years. That is, the leading experts in AI believe that there is a 50% chance that humanity will be fully redundant in 45 years.

This prediction is unimaginable to most of us.But is it realistic? In the next sections I will answer this question looking at the different types of work.But firstly, I will explain a little about recent AI advancements.

Up until recently we were very much in an AI winter (a term coined by relating it to a nuclear winter), where there were distinct phases of hype, followed by disappointment and criticism.The disillusionment was reflected in pessimism by the media, and severe cutbacks in funding, resulting in reduced interest in serious research.

This lull has changed in the last decade or so, with the success of deep learning - an AI paradigm that was inspired by how the brain processes information (in short, artificial neural networks that that process information in parallel, as opposed to the typical serial processing we see in most computer CPUs).

Deep learning and neural networks have been around for some time. However, it is only recently that our computers have been powerful enough to run these algorithms on real-world problems, in real-time.For example, visual object recognition systems of today (e.g., facebooks face recognition system) use what are called convolutional neural networks that mimic how the human visual cortex works. Papers describing this approach started appearing in the early 80s such as with Fukushimas Neocognitron.However, it was not until 2011 that our computers were able to run these algorithms at an appropriate speed to make them useful in practice.

What happened around only 10 years ago, was it was discovered that neural networks could run on computer graphics cards (GPUs - graphics programming units) as these cards were specifically designed to process large amounts of information in parallel - exactly what is needed for artificial neural networks.Most AI researchers these days still use high-performance graphics cards, with there being exponential growth in the capabilities of these cards over time. That is, graphics cards today are 16x more powerful than what they were 10 years ago, and 4x what they were 5 years ago that is they double in computational power every 2.5 years.And with growing interest in the area, we are sure to see ongoing rapid advancements in this technology.

Beyond the fact that computers can now run these large scale networks in real-time, we also have a wealth of large data sources to train them on (n.b., neural networks learn from examples), and available programming platforms such as TensorFlow developed by Google that are openly available to anyone with an interest in machine learning.

As a result of the availability and success of deep learning approaches, AI has officially moved from its supposed winter, to a new season - spring.

What does all this mean for the workforce?Lets continue...

Perhaps one of the low hanging fruits of robotics and AI is in automation replacing repetitive manual labour with machines that can perform the same kind of task cheaply and more efficiently.

An example of this is in Alibabas smart warehouse, where robots perform 70% of the work:

I think the important thing to note when we think of AI replacing human workers, is that they do not have to do the same exact work, in order to make humans redundant.

Consider how Alibaba and Amazon have disrupted the retail sector, with an increasing number of shoppers heading for their screens to make their purchases rather than entering brick-and-mortar retail stores.The outcome is the same (i.e. a consumer making a purchase and receiving a product), but the process itself can be restructured in a way that uses automation to make the process cheaper and more efficient.

For example, Amazon (Prime Air), is trialing a drone delivery system to safely delivery packages to customers in 30 minutes or less disrupting the standard way humans would make similar deliveries:

We are seeing much progress in the ability of machines to perform manual tasks in a wide range of areas, both in and outside of the factory.Take for example the task of laying bricks. Fastbrick, has recently signed a contract in Saudi Arabia to build 55,000 homes by 2022, using automation:

As a glimpse of the future, companies such as Boston Robotics are capable of building robots with similar physical structures to humans, performing tasks that average humans cant:

The point here is that robots in the near future will no doubt be able to replace low-skilled workers, in menial and repetitive tasks, either by performing it in a similar manner, or changing the nature of the work itself, solving the same task in a different but more efficient manner.

I was speaking with a leader only last week, who was about to replace his airport baggage handling staff with machines.And he simply said, the robots will be cheaper and do a better job so why wouldnt he.

And the truth is, this is how these decisions are being made.To gain or maintain a competitive advantage, automation is indeed a rational choice.But what does this mean for the unskilled labourers?

What we currently know is that the gap between the rich and the poor is growing rapidly (e.g., the 3 richest people in the world possess more financial assets than the lowest 48 nations combined):

In the bottom percentiles the number of hours worked has decreased substantially, with the main reason being the demand and supply of skills.

Many argue that although machines will no doubt take on the low-skilled jobs, these workers will simply move to positions where more human-like traits are required (e.g., emotional intelligence and creativity).Will will delve into these areas, to test this assumption in the following sections. But from research such as the above, the current trend has been so far to replace workers without creating the equal number of opportunities elsewhere.

A level up from automation, are jobs or aspects of jobs that require decision-making and problem-solving.

In terms of decision-making, AI is incredibly well-suited for statistical decision-making tasks. That is, given a description of the current situation, categorising the data into appropriate classes. Examples of this include speech-to-text recognition (where the auditory stream is classified into distinct words), language translation (converting one representation to another), object detection (e.g., finding objects or detecting faces in an image), medical diagnoses (e.g., detecting the presence of cancerous cells), exam grading, and modelling consumer behaviour etc. etc.These systems are perfect for scenarios where there is a lot of data that can be used for training the systems, and there are numerous examples (such as those I just listed) where machines now outperform their human counterparts.

I place problem-solving here in a slightly different category to decision-making.Problem-solving is more about how to get to a desired state given the current state, and may involve a number of steps to get there.Navigation is a perfect example of this. And we have seen how well technologies such as google maps have been integrated into our daily lives (e.g., calculating the fastest route given current traffic conditions, and modifying the recommendation should conditions change).

Deep learning has also had a major impact in AI approaches to problem-solving.Take for example chess. In 1997 Deep Blue, a chess-playing computer developed by IBM beat Garry Kasparov, becoming the first computer system to defeat a reigning world champion.This system used a brute force approach, thinking ahead, and evaluating 200 million positions per second. This is quite distinct to how humans experts play chess, that play through intuition rather than thinking through all possible exhaustive moves.

With the advent of deep learning, AI problem-solving has become more human-like.Googles AlphaZero for instance has beaten the worlds best chess-playing computer, teaching itself how to play in under 4 hours.Rather than using brute force AlphaZero uses deep learning (plus a few other tricks) to extract patterns of patterns that it can use to evaluate its next move.Thus, this is similar to human intuition, where it has a feeling how good a move is based on the global context. Similar to human intuition, one drawback of this approach is that it is often impossible to understand "how" the decision was made (as it is due to the combination of millions of features at different levels).

Besides chess, Google has also beaten the world champion at the ancient Chinese game of go.This was a major achievement, as it was foreseen by AI researchers as an incredibly difficult task.In a game of chess, there are on average approximately 35 legal positions that a player can make on each move.By comparison, the average branching factor for Go is 250, making a brute force search intractable. In 2016, AlphaGo won 4-1 against Lee Sedol, widely considered to be the greatest player in the past decade.AplhaGos successor, AlphaGo Zero, described in Nature, is arguably the strongest Player in history.

So in short Again, there is much growing research and success in computer decision-making and problem solving.

When talking about the future of work, there is often an argument that, although machines will replace many jobs, there will always be a space between what AI and humans can do.Accordingly, human work will simply move to areas that involve creativity and emotional intelligence - competencies that machines will never be good at. Lets explore this argument, as it was the topic of my own PhD.

My own PhD work (gosh, around 20 years ago now), was inspired by Douglas Hofstadter and the Fluid Analogies Research Group (FARG)- a team of AI researchers investigating the fundamental processes underlying human perception and creativity.

Many of the models that FARG implemented seem trivial by todays standard, but illustrate the core processes underlying human creativity.

One of the many examples of creativity that they looked at, was the game JUMBLE - a simple newspaper game, where you were required to unscramble the given letters into a real word.Consider the scrambled letters UFOHGT

Now, you are probably asking yourself what this trivial anagram task has to do with creativity.And the answer is EVERYTHING.

While trying to solve this problem, think about HOW you solve it.

Unlike Deep Blue, in solving anagrams, you will not search through every combination.But instead you will create word-like candidates - letter strings that follow the statistical properties of what words generally look like.E.g., you would not start with the letters HG together or FG, as statistically speaking, these are not how typical English words start.

You might instead start by chunking the letters OU FT and GH together, and arrange them in a sequence to create the word GHOUFT.But you discover that this is a non-word, and you pull it apart and try again.

Over time, you will try different combinations of word-like candidates until you come up with a real English word.

The creative aspect of this task lies in the fact that you generated a range of word-like options based on the statistical properties of english.

A demo of this (one of the many demos from my thesis) can be found below:

In short, most creativity can be viewed in this manner. that there exists statistical regularities of things that belong together, and the creative process involves searching through a range of options until you find a global solution that is suitable.For example, music is not a random sequence of notes but has inherent structure, with music composition exploring different combinations of notes that conform to these rules.

With the advent of deep learning, AI now is very good at extracting such statistical regularities from domains, and generating novel examples that follow the statistics of the domain.

An example of this is from Sonys CSL Research Lab that can listen to music, extract the statistical regularities, and generate its own songs in the given style.As an example, the below song was generated in the style of the Beatles:

An example perhaps more illustrative of current advances is Googles drawing bot that is capable of generating photo-realistic images, given a text description. This system was trained on captioned images, and once trained could generate its own images given a text based description.

For example, the following drawing was generated from the query this bird is black and yellow with a short beak (i.e. this bird does not exist in real life, and was generated by the algorithm rather than being retrieved):

This system can generate a range of images, including ordinary pastoral scenes such as grazing livestock, through to more abstract concepts such as a floating double-decker bus.

Another example of this is in computer programming - as this is something schools are focussing on, that they believe will be an essential skill of the future.


Researchers from Rice University havecreated and are refining an applicationthat writes code given a short verbal description from the user.

The software uses deep learning to "read the programmer's mind and predict the program they want."

So, in short... There is major disruption about to potentially occur in this area as well. The future (and as such what we need to be teaching kids to prepare them for it), is very uncertain.

So - to answer the question that I started above.Yes, in the short-term there may be spaces between what humans and machines are capable of, but in the near future these spaces will get smaller and smaller.

In the long-term, I certainly believe creativity is an area that could and will be outsourced to machines (particularly in the technical space of creating new ideas and solutions).

The final bastion that seems to protect humans (and our jobs) from complete redundancy, is our human emotions and emotional intelligence.Many people argue that this is a defining feature of humans that truly segregates us from machines.

Or does it?

If you believe in evolution, you should believe that humans have emotions for a reason - there is some evolutionary benefit.

Thought of in this way, most emotions are definitely here for a reason.They are our internal guidance system that tells us if we are getting things right or wrong.For example, pain and fear are incredibly important, as they prevent us from taking risks that could lead to harm.No doubt such emotions are useful for machines to have as well, and we already see early analogues of them in machines of today (e.g., bump sensors, or sensors to prevent your robot vacuum cleaner from falling down stairs - sensors that prevent them from doing things that could be self-harming).

Ok, so pain is an obviously important signal for machines to have, but what about something more complex like happiness - what could be the evolutionary benefit of that?

Well, I am glad you asked, as it has been part of my own research to look at pleasure centres in the brain, and develop their analogue in robots - yes, indeed happy robots.

You can check out some of my older research on this topic in the video below.In short, one of the many purposes of happiness is that it drives learning (i.e. we are naturally curious, and are as a result active participants in our own learning).

So, hopefully, watching the above video, you will understand the role of emotions, and how they are central to intelligence.So, I do definitely believe that in the near future machines will have their own emotions and drives that will increase in complexity over time. There is no real bastion that will be left standing in the end.

I have made many strong claims in the above text (i.e. that in the near future, all human jobs are in jeopardy), and I am sure that there will be more than a few people that may be skeptical at this point.Possibly because the advances in current AI are not visible in our lives - they are currently hidden away in our factories, mobile phones, and online shopping recommendations. But if you look under the hood, the technology is there, and progressing at an alarming rate.

A possible metaphor is that of the boiling frog - i.e. if you put a frog in boiling water it will jump out immediately, but if you put a frog in cool water that is brought slowly to boil it will not perceive the threat and be boiled alive (not scientifically true, but a nice metaphor).

As humans, we are used to slow and gradual change.In contrast, we are unfamiliar with exponential growth, in that something that we perceive as changing slowly today, may rapidly change tomorrow.As a result, rapid overnight changes are not something we naturally fear. But all the research suggests that advances in AI are following this exponential pattern, and there is a tipping point in the near future where changes will be rapid and unpredictable.

Ray Kurzweil, in his book the age of the spiritual machines charts evidence of this exponential growth, in terms of the increase of calculations per second of computers over time.Given the current GPUs we are currently using for AI, the predictions he made with respect to where we are in 2018 are remarkably accurate.

What is scary about this graph is that if these trends continue, the average PC will have the computational power of the human brain by around 2030, and the computational power of the entire human population in around 2050.

I am not necessarily saying that these predictions are fully accurate, but I do believe that as individuals we are underestimating the rapid changes to our lives that are about to occur.

If you view AI as a species that is evolving, it is evolving at a pace unlike anything we have ever witnessed before, and in the last 10 years, progress has been remarkable.

As an interesting example of this, check out Google Duplex:

There is no doubt that there is a tsunami of change that is about to hit our shores.A tsunami that few people are expecting, with a ferocity and timescale potentially more threatening to our species than climate change.

The danger I believe, is not in the technology itself, but in how we are using it.

If we use AI unchecked for corporate competitive advantage, there is no doubt companies will choose the cheapest and most efficient option, and the employees at the lowest levels will be the first to be hit hard.But over time, it is highly likely that all of our opportunities at all levels will be washed away. And very soon.

But it is a tsunami. I do believe we can channel for the greater good if we choose to.If we dont use it for corporate advantage but instead use it to solve the biggest issues facing humanity such as education, poverty, famine, disease and climate change.

I also fear that this issue will be similar to climate change that the leaders at the top will be reluctant to take action (e.g., it is unfathomable that some current world leaders are still denying that global warming is an issue, despite there being a 97% consensus by climate specialists).

So what can be done?

They say that the Holocaust was allowed to occur in Nazi Germany because the good people sat back and did nothing.Some later justified their inaction claiming that they did not know where the trains were heading. Today, we do not have this excuse. in terms of both climate change and AI, we know exactly where these trains are heading. and these trains contain our children.

I think the major problem we face in stopping or redirecting these trains (i.e. pressuring the government to intervene) is what in the psychological literature is known as bystander apathy - the fact that people in a crowd are less likely to step in and help than individuals witnessing an atrocity alone.

With bystander apathy, people only step in to help when:

1) they notice that something is going on

2) interpret the situation as being an emergency

3) feel that they have a degree of responsibility (i.e. there is no-one else who is better suited)

4) and know what to do to help.

So if it is really up to the people to upward manage our governments (to make sure companies act sustainably and in a way that is beneficial to humanity) - how do we avoid our own bystander apathy?

So if it is really up to the people to upwardly manage our governments (to make sure companies act sustainably and in a way that is beneficial to humanity) - how do we avoid our own bystander apathy?

See the original post:

Artificial Intelligence, and the Future of Work Should We Be Worried? - BBN Times

Written by admin

October 21st, 2021 at 1:46 am

Posted in Alphazero

This AI chess engine aims to help human players rather than defeat them – The Next Web

Posted: January 31, 2021 at 8:53 am

without comments

Artificial intelligence has become so good at chess that its only competition now comes from other computer programs.Indeed, a human hasnt defeated a machine in a chess tournament in 15 years.

Its an impressive technical achievement, but that dominance has also made top-level chess less imaginative, as players now increasingly follow strategies produced by soulless algorithms.

But a newresearch papershows that AI could still make the game better for us puny humans.

The study authors developed a chess enginewith a difference. Unlike most of its predecessors, their system isnt designed to defeat humans. Instead, its programmed to play like them.

[Read: How this company leveraged AI to become the Netflix of Finland]

The researchers believe Maiacould make the game more fun to play. But it could also help us learn from the computer.

So chess becomes a place where we can try understanding human skill through the lens of super-intelligent AI, said study co-author Jon Kleinberg, a professor at Cornell University.

Their system called Maia is a customized version of AlphaZero, a program developed by research lab DeepMind to master chess, Shogi, and Go.

Instead of building Maia to win a game of chess, the model was trained on individualmoves made by humans. Studyco-author Ashton Anderson said this allowed the system to spot what players should work on:

Maia has algorithmically characterized which mistakes are typical of which levels, and therefore which mistakes people should work on and which mistakes they probably shouldnt, because they are still too difficult.

Maia matched the movesof humans more than 50% of the time, and its accuracy grew as the skill level increases.

The researchers said this prediction accuracy is higher than that of Stockfish, the reigning computer world chess champion.

Maia might not be capable of teaching people to conquer AI at chess but it could help beat their fellow humans.

You can read the study paper on the preprint server arXiv.

Published January 27, 2021 18:52 UTC

Excerpt from:

This AI chess engine aims to help human players rather than defeat them - The Next Web

Written by admin

January 31st, 2021 at 8:53 am

Posted in Alphazero

Toronto scientists help create AI-powered bot that can play chess like a human – blogTO

Posted: at 8:53 am

without comments

If you know anything about the intersection of chess and technology, you're likely familiar with IBM's famous "Deep Blue" the first ever computer to beat a reigning (human) world champion at his own game back in 1997.

A lot has happened in the world of artificial intelligence since that time, to the point where humans can no longer even compete with chess engines. They're simply too powerful.

In fact, according to the authors of an exciting new paper on the subject, no human has been able to beat a computer in a chess tournament for more than 15 years.

Given that human beingsdon't generally like to lose, playing chess with a purpose-built bot is nolonger enjoyable. But what if an AI could be trained to play not like a robot, but another person?

Meet Maia, a new "human-like chess engine" that has been trained not to beat people, but to emulate them, developed by researchers at the University of Toronto, Cornell University and Microsoft.

Using the open-source, neural network-based chess engine Leela, which is based on DeepMind'sAlphaZero, the scientists trained Maia usingmillions of actual online human games "with the goal of playing the most human-like moves, instead of being trained on self-play games with the goal of playing the optimal moves."

Nine different Maias were actuallydeveloped to account for varying skill levels, all of them producing different (but incredibly positive) results in terms of how well they could predict the exact moves of human players in actual games.

According to the paper, which is co-authored by U of T's Ashton Anderson,Maia now"predicts human moves at a much higher accuracy than existing engines, and can achieve maximum accuracy when predicting decisions made by players at a specific skill level in a tuneable way."

Maia takes skill level into account when predicting which moves its human opponents will take. Here, the chess enginepredicts that people will stop playing a specific wrong move when they're rated around 1500. Image via the Maia team.

Cool? Certainly and the project is already helping online chess buffs play more enjoyable matches. But the implications of the research actually go far beyond online games.

The root goal in developing Maia, according to the study's authors, was to learn more about how to improve human-AI interaction.

"As artificial intelligence becomes increasingly intelligent in some cases, achieving superhuman performance there is growing potential for humans to learn from and collaborate with algorithms," reads a description of the project from U of T's Computational Social Science Lab.

"However, the ways in which AI systems approach problems are often different from the ways people do, and thus may be uninterpretable and hard to learn from."

The researchers say that a crucial step in bridging the gap between human and machine learning styles is to model "the granular actions that constitute human behavior," as opposed to simply matching the aggregate performance of humans.

In other words, they need to teach the robots not only what we know, but how to think like us.

"Chess been described as the 'fruit fly' of AI research," said Cornell professor and study co-author Jon Kleinberg in a news update published by the American univeristy this week.

"Just as geneticists often care less about the fruit fly itself than its role as a model organism, AI researchers love chess, because it's one of their model organisms. Its a self-contained world you can explore, and it illustrates many of the phenomena that we see in AI more broadly."

Excerpt from:

Toronto scientists help create AI-powered bot that can play chess like a human - blogTO

Written by admin

January 31st, 2021 at 8:53 am

Posted in Alphazero

Scientists say dropping acid can help with social anxiety and alcoholism – The Next Web

Posted: at 8:53 am

without comments

What happens when the pandemic finally ends and hundreds of millions of people whove spent an inordinate amount of time secluded are suddenly launched back into the rat race?

Things will likely never go back to normal, but eventually well find a way to occupy space together again and that could be difficult for people whove developed social anxiety or had setbacks in their treatment due to the unique nature of pandemic isolation.

We couldnt find any actual rats to ask how theyre coping with the race, but a team of laboratory mice might just have the answer: its dropping a bunch of acid and letting nature do its thing.

According to a team of researchers from McGill University, LSD (colloquially known as acid) makes people more social and capable of greater human empathy.

The team figured this out by giving lab mice LSD and then measuring their brain activity. The mice became more social while under the influence. And the positive effects of LSD were immediately nullified when the scientists used bursts of light to interrupt the chemical processes thus rendering the mice immediately sober.

The researchers work led to novel insight into how LSD causes a cascade effect of receptor and synapse activity that ultimately seems to kick-start neurotypical feelings of empathy and social inclination.

Due to the nature of the specific chemical reactions concurring in the brain upon the consumption of LSD, it would appear as though its a strong candidate for the potential treatment of myriad mental illnesses and for those with autism spectrum disorder.

Per the teams research paper:

These results indicate that LSD selectively enhances SB by potentiating mPFC excitatory transmission through 5-HT2A/AMPA receptors and mTOR signaling. The activation of 5-HT2A/AMPA/mTORC1 in the mPFC by psychedelic drugs should be explored for the treatment of mental diseases with SB impairments such as autism spectrum disorder and social anxiety disorder.

Quick take: Scientists have understood the effect LSD has on mood receptors in the brain for decades. Whats new here is that we now know how those interactions cause other interactions that create whats essentially a system for increasing empathy or decreasing social anxiety.

Recent research on LSD, cannabis, and psilocybin (shrooms) indicates each has myriad uses for combating and treating mental illness and other disorders related to neurotypical receptor and synapse regulation.

The McGill teams research on LSD, for example, indicates it could prove useful to fight the harmful effects of alcoholism where people are at increased risk of developingsocial anxiety due toaddiction, thus further isolating themselves from others.

This latest study is important in that it drives home what decades of research and millennia of anecdotal evidence already tells us: Some drugs have the potential to do good.

And if we could study them like rational humans instead of allowing politicians to make it almost impossible for researchers to conduct controlled, long term studies on so-called banned substances the world would be a better place.

If you think this is interesting, check out this piece on Neural from earlier today. Where the study in the article youve just read says LSD can amplify empathy and reduce social anxiety, this one shows how empathy happens in a theory of the mind that can be identified down to the single-neuron level.

Read next: Zuckerberg promises Facebook will show less political content from now on

Read more here:

Scientists say dropping acid can help with social anxiety and alcoholism - The Next Web

Written by admin

January 31st, 2021 at 8:53 am

Posted in Alphazero

AI has almost solved one of biologys greatest challenges how protein unfolds – ThePrint

Posted: December 14, 2020 at 1:55 am

without comments

Text Size: A- A+

Solving what biologists call the protein-folding problem is a big deal. Proteins are the workhorses of cells and are present in all living organisms. They are made up of long chains of amino acids and are vital for the structure of cells and communication between them as well as regulating all of the chemistry in the body.

This week, the Google-owned artificial intelligence company DeepMind demonstrated a deep-learning program called AlphaFold2, which experts are calling a breakthrough toward solving the grand challenge of protein folding.

Proteins are long chains of amino acids linked together like beads on a string. But for a protein to do its job in the cell, it must fold a process of twisting and bending that transforms the molecule into a complex three-dimensional structure that can interact with its target in the cell. If the folding is disrupted, then the protein wont form the correct shape and it wont be able to perform its job inside the body. This can lead to disease as is the case in a common disease like Alzheimers, and rare ones like cystic fibrosis.

Deep learning is a computational technique that uses the often hidden information contained in vast datasets to solve questions of interest. Its been used widely in fields such as games, speech and voice recognition, autonomous cars, science and medicine.

I believe that tools like AlphaFold2 will help scientists to design new types of proteins, ones that may, for example, help break down plastics and fight future viral pandemics and disease.

I am a computational chemist and author of the book The State of Science. My students and I study the structure and properties of fluorescent proteins using protein-folding computer programs based on classical physics.

After decades of study by thousands of research groups, these protein-folding prediction programs are very good at calculating structural changes that occur when we make small alterations to known molecules.

But they havent adequately managed to predict how proteins fold from scratch. Before deep learning came along, the protein-folding problem seemed impossibly hard, and it seemed poised to frustrate computational chemists for many decades to come.

The sequence of the amino acids which is encoded in DNA defines the proteins 3D shape. The shape determines its function. If the structure of the protein changes, it is unable to perform its function. Correctly predicting protein folds based on the amino acid sequence could revolutionize drug design, and explain the causes of new and old diseases.

All proteins with the same sequence of amino acid building blocks fold into the same three-dimensional form, which optimizes the interactions between the amino acids. They do this within milliseconds, although they have an astronomical number of possible configurations available to them about 10 to the power of 300. This massive number is what makes it hard to predict how a protein folds even when scientists know the full sequence of amino acids that go into making it. Previously predicting the structure of protein from the amino acid sequence was impossible. Protein structures were experimentally determined, a time-consuming and expensive endeavor.

Once researchers can better predict how proteins fold, theyll be able to better understand how cells function and how misfolded proteins cause disease. Better protein prediction tools will also help us design drugs that can target a particular topological region of a protein where chemical reactions take place.

Also read: Diabetics sugar can rise based on how much they think theyre having, Harvard study finds

The success of DeepMinds protein-folding prediction program, called AlphaFold, is not unexpected. Other deep-learning programs written by DeepMind have demolished the worlds best chess, Go and poker players.

In 2016 Stockfish-8, an open-source chess engine, was the worlds computer chess champion. It evaluated 70 million chess positions per second and had centuries of accumulated human chess strategies and decades of computer experience to draw upon. It played efficiently and brutally, mercilessly beating all its human challengers without an ounce of finesse. Enter deep learning.

On Dec. 7, 2017, Googles deep-learning chess program AlphaZero thrashed Stockfish-8. The chess engines played 100 games, with AlphaZero winning 28 and tying 72. It didnt lose a single game. AlphaZero did only 80,000 calculations per second, as opposed to Stockfish-8s 70 million calculations, and it took just four hours to learn chess from scratch by playing against itself a few million times and optimizing its neural networks as it learned from its experience.

AlphaZero didnt learn anything from humans or chess games played by humans. It taught itself and, in the process, derived strategies never seen before. In a commentary in Science magazine, former world chess champion Garry Kasparov wrote that by learning from playing itself, AlphaZero developed strategies that reflect the truth of chess rather than reflecting the priorities and prejudices of the programmers. Its the embodiment of the clich work smarter, not harder.

Every two years, the worlds top computational chemists test the abilities of their programs to predict the folding of proteins and compete in the Critical Assessment of Structure Prediction (CASP) competition.

In the competition, teams are given the linear sequence of amino acids for about 100 proteins for which the 3D shape is known but hasnt yet been published; they then have to compute how these sequences would fold. In 2018 AlphaFold, the deep-learning rookie at the competition, beat all the traditional programs but barely.

Two years later, on Monday, it was announced that Alphafold2 had won the 2020 competition by a healthy margin. It whipped its competitors, and its predictions were comparable to the existing experimental results determined through gold standard techniques like X-ray diffraction crystallography and cryo-electron microscopy. Soon I expect AlphaFold2 and its progeny will be the methods of choice to determine protein structures before resorting to experimental techniques that require painstaking, laborious work on expensive instrumentation.

One of the reasons for AlphaFold2s success is that it could use the Protein Database, which has over 170,000 experimentally determined 3D structures, to train itself to calculate the correctly folded structures of proteins.

The potential impact of AlphaFold can be appreciated if one compares the number of all published protein structures approximately 170,000 with the 180 million DNA and protein sequences deposited in the Universal Protein Database. AlphaFold will help us sort through treasure troves of DNA sequences hunting for new proteins with unique structures and functions.

As with the chess and Go programs AlphaZero and AlphaGo we dont exactly know what the AlphaFold2 algorithm is doing and why it uses certain correlations, but we do know that it works.

Besides helping us predict the structures of important proteins, understanding AlphaFolds thinking will also help us gain new insights into the mechanism of protein folding.

[Deep knowledge, daily. Sign up for The Conversations newsletter.]

One of the most common fears expressed about AI is that it will lead to large-scale unemployment. AlphaFold still has a significant way to go before it can consistently and successfully predict protein folding.

However, once it has matured and the program can simulate protein folding, computational chemists will be integrally involved in improving the programs, trying to understand the underlying correlations used, and applying the program to solve important problems such as the protein misfolding associated with many diseases such as Alzheimers, Parkinsons, cystic fibrosis and Huntingtons disease.

AlphaFold and its offspring will certainly change the way computational chemists work, but it wont make them redundant. Other areas wont be as fortunate. In the past robots were able to replace humans doing manual labor; with AI, our cognitive skills are also being challenged.

Marc Zimmer, Professor of Chemistry, Connecticut College

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Also read: Egg sales have skyrocketed during Covid pandemic and other eggy facts

Subscribe to our channels on YouTube & Telegram

Why news media is in crisis & How you can fix it

India needs free, fair, non-hyphenated and questioning journalism even more as it faces multiple crises.

But the news media is in a crisis of its own. There have been brutal layoffs and pay-cuts. The best of journalism is shrinking, yielding to crude prime-time spectacle.

ThePrint has the finest young reporters, columnists and editors working for it. Sustaining journalism of this quality needs smart and thinking people like you to pay for it. Whether you live in India or overseas, you can do it here.

Support Our Journalism

Read the original here:

AI has almost solved one of biologys greatest challenges how protein unfolds - ThePrint

Written by admin

December 14th, 2020 at 1:55 am

Posted in Alphazero

Facebook AI Introduces ‘ReBeL’: An Algorithm That Generalizes The Paradigm Of Self-Play Reinforcement Learning And Search To Imperfect-Information…

Posted: at 1:55 am

without comments

Most AI systems excel in generating specific responses to a particular problem. Today, AI can outperform humans in various fields. For AI to do any task it is presented with; it needs to generalize, learn, and understand new situations as they occur without supplementary guidance. However, as humans can recognize chess and Poker both as games in the broadest sense, teaching a single AI to play both is challenging.

Perfect-Information games versus Imperfect-Information games

AI systems are relatively successful at mastering perfect-information games like chess, where nothing is hidden to either player. Each player can see the entire board and all possible moves in all instances. With bots like AlphaZero, AI can even combine reinforcement learning with search (RL+Search) to teach themselves to master these games from scratch.

Unlike perfect-information games and single-agent settings, imperfect-information games have a critical challenge that an actions value may depend on their chosen probability. Therefore, the team states that it is also crucial to include the probability that different sequences of actions occurred and not just the sequences of actions alone.


Facebook has recently introduced Recursive Belief-based Learning (ReBeL). It is a general RL+Search algorithm that works in all two-player zero-sum games, including imperfect-information games. ReBeL grows on the RL+Search algorithms that have proved successful in perfect-information games. However, unlike past AIs, ReBeL makes decisions by factoring in the probability distribution of different views each player might have about the games current state, which is called a public belief state (PBS). For example, ReBeL can assess the chances that its poker opponent thinks it has.

Former RL+Search algorithms break down in imperfect-information games like Poker, where not complete information is known (for example, players keep their cards secret in Poker). These algorithms give a fixed value to each action regardless of whether the action is chosen. For instance, in chess, a right step is good irrespective of whether it is chosen frequently or rarely. But in games like Poker, the more a player bluffs, its value goes down as opponents can alter their strategy to call more of those bluffs. Thus Pluribus poker bot is trained on an approach that uses search during actual gameplay and not before.

ReBeL can treat imperfect-information games similar to perfect-information games by accounting for the views of each player. Facebook has developed a modified RL+Search algorithm that ReBeL can leverage to work with the higher-dimensional state and action range of imperfect-information games.

Experiments show that ReBeL is efficient in large-scale two-player zero-sum imperfect-information games such as Liars Dice and Poker. ReBeL achieves superhuman performance by even defeating a top human professional in the benchmark game of heads-up no-limit Texas Hold em.

Several works have occurred before to achieve the same. However, ReBeL executes it using considerably less expert domain knowledge than any previous poker AI. This is a crucial step to building a generalized AI that can solve complex real-world problems involving hidden information like negotiations, fraud detection, cybersecurity, etc.


ReBeL is the first AI to empower RL+Search in imperfect-information games. However, there are some limitations to its current implementation, as listed below:

Nevertheless, ReBeL achieves low exploitability in benchmark games and is a significant start toward creating more general AI algorithms. To promote further research, Facebook has open-sourced the implementation of ReBeL for Liars Dice.

GitHub: (For ReBeL for Liars Dice)


Related Paper:

Read the original here:

Facebook AI Introduces 'ReBeL': An Algorithm That Generalizes The Paradigm Of Self-Play Reinforcement Learning And Search To Imperfect-Information...

Written by admin

December 14th, 2020 at 1:55 am

Posted in Alphazero

When 3 is greater than 5 – Chessbase News

Posted: October 22, 2020 at 9:59 pm

without comments

10/18/2020 Star columnist Jon Speelman explores the exchange sacrifice. Speelman shares five illustrative examples to explain in which conditions giving up a rook for a minor piece is a good trade. As a general rule and in fact (almost all?) of the time you need other pieces on the board for an exchange sacrifice to work. | Pictured: Mikhail Tal and Tigran Petrosian following a post-mortem analysis at the 1961 European Team Championship in Oberhausen | Photo: Gerhard Hund

ChessBase 15 - Mega package

Find the right combination! ChessBase 15 program + new Mega Database 2020 with 8 million games and more than 80,000 master analyses. Plus ChessBase Magazine (DVD + magazine) and CB Premium membership for 1 year!


[Note that Jon Speelman also looks at the content of the article in video format, here embedded at the end of the article.]

During the Norway tournament, I streamed commentarya couple of times myself at, but mainly listened to the official commentaryby Vladimir Kramnik and Judit Polgar.

Both were very interesting, and Kramnik in particular has a chess aesthetic which I very much like. In his prime a powerhouse positional player with superb endgame technique, he started life much more tactically and his instinct is to sacrifice for the initiative whenever possible, especially the exchange: an approach which, after defence seemed to triumph under traditional chess engines, has been given a new lease of life by Alpha Zero.

So I thought today that Id look at some nice exchange sacrifices, but first a moment from Norway where I was actually a tad disappointed by a winning sacrifice.

At the end of a beautiful positional game, which has been annotated here in Game of the Week, Carlsen finished off with the powerful


and after

42...Qxe8 43.Qh6+ Kg8 44.Qxg6+ Kh8 45.Nf6

Tari resigned

Of course, I would have played Re8 myself in a game if Id seen it, but I was hoping from an aesthetic perspective that Carlsen would complete this real masterclass and masterpiece with a nice zugzwang.

You start with c4 to prevent 42.f3 c4, creating some very slight confusion and then it goes:

42.c4 Kg8 43.f3

And for example: 43...Qd7 44.Qh6 Qe6 45.Kg3 fxe4 (45...Rg7 46.Nf6+ Kf7 47.Qh8 Qe7 48.Kg2) 46.dxe4 Rf4 47.Nxf4 exf4+ 48.Kxf4 Qf7+ 49.Kg3 Qg7 50.Qxg7+ Kxg7 51.Rxf8

Black can also try43...Rh7

and here after 44.Rxf8+ Kg7

as the engine pointed out to me, its best to use the Re8 trick:

45.Qxh7+! (45.Rf6 is much messier) 45...Kxh7 46.Re8!

Mega Database 2020

The ChessBase Mega Database 2020 is the premiere chess database with over eight million games from 1560 to 2019 in high quality. Packing more than 85,000 annotated games, Mega 2020 contains the worlds largest collection of high-class analysed games. Train like a pro! Prepare for your opponents with ChessBase and the Mega Database 2020. Let grandmasters explain how to best handle your favorite variations, improve your repertoire and much more.

The black queen is trapped.

For todays examples I used my memory and the ChessBase search mask when I couldnt track down a game exactly. For instance,for the first one byBotvinnik [pictured], I set him as Black with 0-1, disabled ignoring colours, and put Rd4 e5 c5 on the board which turned out to identify the single game I wanted a hole in 1!I also asked my stream on Thursday for any examples, and one of my stalwarts, a Scottish Frenchman, found me Reshevsky v Petrosian (I couldnt remember offhand who Petrosians opponent was) and drew my attention to the beautiful double exchange sacrifice by Erwin L'Ami from Wijk aan Zee B.

Before the games themselves, which are in chronological order,it might be worthwhile to consider what makes an exchange sacrifice successful. Whole books have been written on this and Im certainly not going to be able to go into serious detail. But a couple of points:

The need for extra pieces applies particularly to endgames. For instance,this diagram should definitely be lost for Black:

Its far from trivial, but as a general schema the white king should be able to advance right into Blacks guts and then White can do things with his pawns. Something like get Ke7 and Rf6, then g4 exchanging pawns if Black has played ...h5. Play f5, move the rook, play f6+, and arrange to play Rxf7.

But if you add a pair of rooks then it becomes enormously difficult. And indeed I really dont know whether God would beat God.

Select an entry from the list to switch between games

Master Class Vol.11: Vladimir Kramnik

This DVD allows you to learn from the example of one of the best players in the history of chess and from the explanations of the authors (Pelletier, Marin, Mller and Reeh) how to successfully organise your games strategically, consequently how to keep y

Read more:

When 3 is greater than 5 - Chessbase News

Written by admin

October 22nd, 2020 at 9:59 pm

Posted in Alphazero

AlphaZero – Wikipedia

Posted: October 17, 2020 at 10:54 am

without comments

Game-playing artificial intelligence

AlphaZero is a computer program developed by artificial intelligence research company DeepMind to master the games of chess, shogi and go. This algorithm uses an approach similar to AlphaGo Zero.

On December 5, 2017, the DeepMind team released a preprint introducing AlphaZero, which within 24 hours of training achieved a superhuman level of play in these three games by defeating world-champion programs Stockfish, elmo, and the 3-day version of AlphaGo Zero. In each case it made use of custom tensor processing units (TPUs) that the Google programs were optimized to use.[1] AlphaZero was trained solely via "self-play" using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks, all in parallel, with no access to opening books or endgame tables. After four hours of training, DeepMind estimated AlphaZero was playing at a higher Elo rating than Stockfish 8; after 9 hours of training, the algorithm defeated Stockfish 8 in a time-controlled 100-game tournament (28 wins, 0 losses, and 72 draws).[1][2][3] The trained algorithm played on a single machine with four TPUs.

DeepMind's paper on AlphaZero was published in the journal Science on 7 December 2018.[4] In 2019 DeepMind published a new paper detailing MuZero, a new algorithm able to generalise on AlphaZero work playing both Atari and board games without knowledge of the rules or representations of the game.[5]

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:[1]

Comparing Monte Carlo tree search searches, AlphaZero searches just 80,000 positions per second in chess and 40,000 in shogi, compared to 70 million for Stockfish and 35 million for elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more selectively on the most promising variation.[1]

AlphaZero was trained solely via self-play, using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks. In parallel, the in-training AlphaZero was periodically matched against its benchmark (Stockfish, elmo, or AlphaGo Zero) in brief one-second-per-move games to determine how well the training was progressing. DeepMind judged that AlphaZero's performance exceeded the benchmark after around four hours of training for Stockfish, two hours for elmo, and eight hours for AlphaGo Zero.[1]

In AlphaZero's chess match against Stockfish 8 (2016 TCEC world champion), each program was given one minute per move. Stockfish was allocated 64 threads and a hash size of 1 GB,[1] a setting that Stockfish's Tord Romstad later criticized as suboptimal.[6][note 1] AlphaZero was trained on chess for a total of nine hours before the match. During the match, AlphaZero ran on a single machine with four application-specific TPUs. In 100 games from the normal starting position, AlphaZero won 25 games as White, won 3 as Black, and drew the remaining 72.[8] In a series of twelve, 100-game matches (of unspecified time or resource constraints) against Stockfish starting from the 12 most popular human openings, AlphaZero won 290, drew 886 and lost 24.[1]

AlphaZero was trained on shogi for a total of two hours before the tournament. In 100 shogi games against elmo (World Computer Shogi Championship 27 summer 2017 tournament version with YaneuraOu 4.73 search), AlphaZero won 90 times, lost 8 times and drew twice.[8] As in the chess games, each program got one minute per move, and elmo was given 64 threads and a hash size of 1GB.[1]

After 34 hours of self-learning of Go and against AlphaGo Zero, AlphaZero won 60 games and lost 40.[1][8]

DeepMind stated in its preprint, "The game of chess represented the pinnacle of AI research over several decades. State-of-the-art programs are based on powerful engines that search many millions of positions, leveraging handcrafted domain expertise and sophisticated domain adaptations. AlphaZero is a generic reinforcement learning algorithm originally devised for the game of go that achieved superior results within a few hours, searching a thousand times fewer positions, given no domain knowledge except the rules."[1] DeepMind's Demis Hassabis, a chess player himself, called AlphaZero's play style "alien": It sometimes wins by offering counterintuitive sacrifices, like offering up a queen and bishop to exploit a positional advantage. "It's like chess from another dimension."[9]

Given the difficulty in chess of forcing a win against a strong opponent, the +28 0 =72 result is a significant margin of victory. However, some grandmasters, such as Hikaru Nakamura and Komodo developer Larry Kaufman, downplayed AlphaZero's victory, arguing that the match would have been closer if the programs had access to an opening database (since Stockfish was optimized for that scenario).[10] Romstad additionally pointed out that Stockfish is not optimized for rigidly fixed-time moves and the version used is a year old.[6][11]

Similarly, some shogi observers argued that the elmo hash size was too low, that the resignation settings and the "EnteringKingRule" settings (cf. shogi Entering King) may have been inappropriate, and that elmo is already obsolete compared with newer programs.[12][13]

Papers headlined that the chess training took only four hours: "It was managed in little more than the time between breakfast and lunch."[2][14]Wired hyped AlphaZero as "the first multi-skilled AI board-game champ".[15] AI expert Joanna Bryson noted that Google's "knack for good publicity" was putting it in a strong position against challengers. "It's not only about hiring the best programmers. It's also very political, as it helps make Google as strong as possible when negotiating with governments and regulators looking at the AI sector."[8]

Human chess grandmasters generally expressed excitement about AlphaZero. Danish grandmaster Peter Heine Nielsen likened AlphaZero's play to that of a superior alien species.[8] Norwegian grandmaster Jon Ludvig Hammer characterized AlphaZero's play as "insane attacking chess" with profound positional understanding.[2] Former champion Garry Kasparov said "It's a remarkable achievement, even if we should have expected it after AlphaGo."[10][16]

Grandmaster Hikaru Nakamura was less impressed, and stated "I don't necessarily put a lot of credibility in the results simply because my understanding is that AlphaZero is basically using the Google supercomputer and Stockfish doesn't run on that hardware; Stockfish was basically running on what would be my laptop. If you wanna have a match that's comparable you have to have Stockfish running on a supercomputer as well."[7]

Top US correspondence chess player Wolff Morrow was also unimpressed, claiming that AlphaZero would probably not make the semifinals of a fair competition such as TCEC where all engines play on equal hardware. Morrow further stated that although he might not be able to beat AlphaZero if AlphaZero played drawish openings such as the Petroff Defence, AlphaZero would not be able to beat him in a correspondence chess game either.[17]

Motohiro Isozaki, the author of YaneuraOu, noted that although AlphaZero did comprehensively beat elmo, the rating of AlphaZero in shogi stopped growing at a point which is at most 100~200 higher than elmo. This gap is not that high, and elmo and other shogi software should be able to catch up in 12 years.[18]

DeepMind addressed many of the criticisms in their final version of the paper, published in December 2018 in Science.[4] They further clarified that AlphaZero was not running on a supercomputer; it was trained using 5,000 tensor processing units (TPUs), but only ran on four TPUs and a 44-core CPU in its matches.[19]

In the final results, Stockfish version 8 ran under the same conditions as in the TCEC superfinal: 44 CPU cores, Syzygy endgame tablebases, and a 32GB hash size. Instead of a fixed time control of one move per minute, both engines were given 3 hours plus 15 seconds per move to finish the game. In a 1000-game match, AlphaZero won with a score of 155 wins, 6 losses, and 839 draws. DeepMind also played a series of games using the TCEC opening positions; AlphaZero also won convincingly.

Similar to Stockfish, Elmo ran under the same conditions as in the 2017 CSA championship. The version of Elmo used was WCSC27 in combination with YaneuraOu 2017 Early KPPT 4.79 64AVX2 TOURNAMENT. Elmo operated on the same hardware as Stockfish: 44 CPU cores and a 32GB hash size. AlphaZero won 98.2% of games when playing black (which plays first in shogi) and 91.2% overall.

Human grandmasters were generally impressed with AlphaZero's games against Stockfish.[20] Former world champion Garry Kasparov said it was a pleasure to watch AlphaZero play, especially since its style was open and dynamic like his own.[21][22]

In the chess community, Komodo developer Mark Lefler called it a "pretty amazing achievement", but also pointed out that the data was old, since Stockfish had gained a lot of strength since January 2018 (when Stockfish 8 was released). Fellow developer Larry Kaufman said AlphaZero would probably lose a match against the latest version of Stockfish, Stockfish 10, under Top Chess Engine Championship (TCEC) conditions. Kaufman argued that the only advantage of neural networkbased engines was that they used a GPU, so if there was no regard for power consumption (e.g. in an equal-hardware contest where both engines had access to the same CPU and GPU) then anything the GPU achieved was "free". Based on this, he stated that the strongest engine was likely to be a hybrid with neural networks and standard alphabeta search.[23]

AlphaZero inspired the computer chess community to develop Leela Chess Zero, using the same techniques as AlphaZero. Leela contested several championships against Stockfish, where it showed similar strength.[24]

In 2019 DeepMind published MuZero, a unified system that played excellent chess, shogi, and go, as well as games in the Atari Learning Environment, without being pre-programmed with their rules.[25][26]

The match results by themselves are not particularly meaningful because of the rather strange choice of time controls and Stockfish parameter settings: The games were played at a fixed time of 1 minute/move, which means that Stockfish has no use of its time management heuristics (lot of effort has been put into making Stockfish identify critical points in the game and decide when to spend some extra time on a move; at a fixed time per move, the strength will suffer significantly). The version of Stockfish used is one year old, was playing with far more search threads than has ever received any significant amount of testing, and had way too small hash tables for the number of threads. I believe the percentage of draws would have been much higher in a match with more normal conditions.[7]


AlphaZero - Wikipedia

Written by admin

October 17th, 2020 at 10:54 am

Posted in Alphazero

AlphaZero: Shedding new light on chess, shogi, and Go …

Posted: at 10:54 am

without comments

As with Go, we are excited about AlphaZeros creative response to chess, which has been a grand challenge for artificial intelligence since the dawn of the computing age with early pioneers including Babbage, Turing, Shannon, and von Neumann all trying their hand at designing chess programs. But AlphaZero is about more than chess, shogi or Go. To create intelligent systems capable of solving a wide range of real-world problems we need them to be flexible and generalise to new situations. While there has been some progress towards this goal, it remains a major challenge in AI research with systems capable of mastering specific skills to a very high standard, but often failing when presented with even slightly modified tasks.

AlphaZeros ability to master three different complex games and potentially any perfect information game is an important step towards overcoming this problem. It demonstrates that a single algorithm can learn how to discover new knowledge in a range of settings. And, while it is still early days, AlphaZeros creative insights coupled with the encouraging results we see in other projects such as AlphaFold, give us confidence in our mission to create general purpose learning systems that will one day help us find novel solutions to some of the most important and complex scientific problems.

This work was done by David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis.

Follow this link:

AlphaZero: Shedding new light on chess, shogi, and Go ...

Written by admin

October 17th, 2020 at 10:54 am

Posted in Alphazero

AlphaGo Zero – Wikipedia

Posted: at 10:54 am

without comments

Artificial intelligence that plays Go

AlphaGo Zero is a version of DeepMind's Go software AlphaGo. AlphaGo's team published an article in the journal Nature on 19 October 2017, introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version.[1] By playing games against itself, AlphaGo Zero surpassed the strength of AlphaGo Lee in three days by winning 100 games to 0, reached the level of AlphaGo Master in 21 days, and exceeded all the old versions in 40 days.[2]

Training artificial intelligence (AI) without datasets derived from human experts has significant implications for the development of AI with superhuman skills because expert data is "often expensive, unreliable or simply unavailable."[3]Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge".[4]David Silver, one of the first authors of DeepMind's papers published in Nature on AlphaGo, said that it is possible to have generalised AI algorithms by removing the need to learn from humans.[5]

Google later developed AlphaZero, a generalized version of AlphaGo Zero that could play chess and Shgi in addition to Go. In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on an Elo scale. AlphaZero also defeated a top chess program (Stockfish) and a top Shgi program (Elmo).[6][7]

AlphaGo Zero's neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers. Only four TPUs were used for inference. The neural network initially knew nothing about Go beyond the rules. Unlike earlier versions of AlphaGo, Zero only perceived the board's stones, rather than having some rare human-programmed edge cases to help recognize unusual Go board positions. The AI engaged in reinforcement learning, playing against itself until it could anticipate its own moves and how those moves would affect the game's outcome.[8] In the first three days AlphaGo Zero played 4.9 million games against itself in quick succession.[9] It appeared to develop the skills required to beat top humans within just a few days, whereas the earlier AlphaGo took months of training to achieve the same level.[10]

For comparison, the researchers also trained a version of AlphaGo Zero using human games, AlphaGo Master, and found that it learned more quickly, but actually performed more poorly in the long run.[11] DeepMind submitted its initial findings in a paper to Nature in April 2017, which was then published in October 2017.[1]

The hardware cost for a single AlphaGo Zero system in 2017, including the four TPUs, has been quoted as around $25 million.[12]

According to Hassabis, AlphaGo's algorithms are likely to be of the most benefit to domains that require an intelligent search through an enormous space of possibilities, such as protein folding or accurately simulating chemical reactions.[13] AlphaGo's techniques are probably less useful in domains that are difficult to simulate, such as learning how to drive a car.[14] DeepMind stated in October 2017 that it had already started active work on attempting to use AlphaGo Zero technology for protein folding, and stated it would soon publish new findings.[15][16]

AlphaGo Zero was widely regarded as a significant advance, even when compared with its groundbreaking predecessor, AlphaGo. Oren Etzioni of the Allen Institute for Artificial Intelligence called AlphaGo Zero "a very impressive technical result" in "both their ability to do itand their ability to train the system in 40 days, on four TPUs".[8]The Guardian called it a "major breakthrough for artificial intelligence", citing Eleni Vasilaki of Sheffield University and Tom Mitchell of Carnegie Mellon University, who called it an impressive feat and an outstanding engineering accomplishment" respectively.[14]Mark Pesce of the University of Sydney called AlphaGo Zero "a big technological advance" taking us into "undiscovered territory".[17]

Gary Marcus, a psychologist at New York University, has cautioned that for all we know, AlphaGo may contain "implicit knowledge that the programmers have about how to construct machines to play problems like Go" and will need to be tested in other domains before being sure that its base architecture is effective at much more than playing Go. In contrast, DeepMind is "confident that this approach is generalisable to a large number of domains".[9]

In response to the reports, South Korean Go professional Lee Sedol said, "The previous version of AlphaGo wasnt perfect, and I believe thats why AlphaGo Zero was made." On the potential for AlphaGo's development, Lee said he will have to wait and see but also said it will affect young Go players. Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGos playing style. "At first, it was hard to understand and I almost felt like I was playing against an alien. However, having had a great amount of experience, Ive become used to it," Mok said. "We are now past the point where we debate the gap between the capability of AlphaGo and humans. Its now between computers." Mok has reportedly already begun analyzing the playing style of AlphaGo Zero along with players from the national team. "Though having watched only a few matches, we received the impression that AlphaGo Zero plays more like a human than its predecessors," Mok said.[18] Chinese Go professional, Ke Jie commented on the remarkable accomplishments of the new program: "A pure self-learning AlphaGo is the strongest. Humans seem redundant in front of its self-improvement."[19]

Future of Go Summit

89:11 against AlphaGo Master

On 5 December 2017, DeepMind team released a preprint on arXiv, introducing AlphaZero, a program using generalized AlphaGo Zero's approach, which achieved within 24 hours a superhuman level of play in chess, shogi, and Go, defeating world-champion programs, Stockfish, Elmo, and 3-day version of AlphaGo Zero in each case.[6]

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:[6]

An open source program, Leela Zero, based on the ideas from the AlphaGo papers is available. It uses a GPU instead of the TPUs recent versions of AlphaGo rely on.


AlphaGo Zero - Wikipedia

Written by admin

October 17th, 2020 at 10:54 am

Posted in Alphazero

Page 1123