Time delay Neural Network MATLAB Code
How do animals establish associations between cues, actions and rewards when rewards come sometime later after the actions that caused them?
The research in the paper “Solving the Distal Reward Problem with Rare Correlations”(1) proposes a new type of computation based on the rarity of correlations. Some neural events that happen rarely are selected to leave a trace which can be detected later in time when a reward occurs. The study shows how the neural models is capable of classical and operant conditioning with delayed rewards.
The PDF of the paper is available for download. Support material include the Matlab code to reproduce the experiments and a short video.
Download PDF of Solving the Distal Reward Problem with Rare Correlations
Download the Matlab code and support video
(1) Andrea Soltoggio and Jochen Steil, Solving the Distal Reward Problem with Rare Correlations, Neural Computation, 2013
How the idea came about
The main idea was inspired by the work of E. Izhikevich who, in 2007, published a computational model to solve the distal reward problem using neuromodulation, eligibility traces and spiking neurons with STDP. A year later, at the Artificial Life Conference in Winchester, UK, a colleague of mine, Chrisantha Fernando, pointed out that work to me . At that point I had been working more than two years on neuromodulation and reward-based learning. Izhikevich’s work was really inspirational, but in contrast to his position, it was soon clear to me that spiking neurons were not crucial and that something else was driving the learning. This thought hung in the back of my mind for a few years until, in the summer of 2011, I met Paul Tonelli, a young researcher working on the evolution of plasticity and learning. We discussed the topic of the distal reward learning in the noisy setting of a Dublin’s pub. The suspicion that rare events (or correlations) were a pivotal part of the learning led me to prove it. A week later I had the first version of the algorithm. Jochen Steil, the leader of my research group, saw the significance of the principle in computational models of learning and robotics. Discussions with him led to further refinements of the algorithm. All simulations and paper were ready in December 2011. Then the long negotiations with the reviewers started and the paper is schedule appear in April 2013 in Neural Computation.
Source: Andrea Soltoggio
You might also like:
Look.............2008-10-16 21:37:03 by 1pushkick
These guys don't know JackS***. Really.
the stock market is an independent creature creation. It is its own entity and is dependent on nothing but the people that control it and it alone. It is not dependent on the companies themselves, the FED, the banks, or anything else.
It is a GAME. It is that simple. It is a game of pattern and momentum. The charts tells you all you need to know.
Lets see what happens. It is and ebb and flow thing. The ebb is ready to flow.
There are many people who need religion as2006-10-07 11:32:49 by Oakland_43
A crutch. Without superstition and magic beliefs, they can't face their lives and often turn to drugs, alcoholism, crime and/or compulsive sexual activity.
As far back as 2500 years ago, Aristotle taught that our "religion" should be the study of nature and Confucius taught a similar thing.
But the majority of people will only study tea leaves at the bottom of cups, whether those cups be stock market charts which might bring them fortune, or Surviver programs which they think can teach them to live, or the expressions on the faces of their ministers which they think can bring them peace.
Arguing with these people is futile. They have exchanged solemn vows with hypocrisy, lies and stupidity.
That doesn't mean that much profound wisdom can't be found in Bibles,...
The Korean War, which started in 1950. When it b2006-07-27 11:47:41 by ----
The Korean War, which started in 1950. When it became clear that the West would have to go to the aid of South Korea to defend it from communist troops in the north, share prices fell in the US by 13 per cent, and to a slightly lesser extent in London. There is no example in recent history when the bursting of a stock market bubble has so closely preceded the outbreak of war. And while the statisticians and military planners can spend forever poring over charts and maps, there is another problem being posed by financial analysts: forget about the prospect of war for a moment, and consider the possibility that the excesses of recent years have not fully unwound.
But the question of valuation levels for share prices is one that refuses to go away. As has been pointed out time and...
Best Bet, Proudest Monkey2009-02-26 22:33:50 by jonahsbelly
The 2009 forecast is ( Dow Jones Forecast 2009 - 20th Jan 2009) - In Summary , I do not know at precisely what price level the Dow will make a low during 2009, my best estimate at this time is 6,600, but I am expecting that it will mark the start of a multi-year bull market that will eventually make 2008-2009's price action appear as a mere minor blip, much as the 1987 crash appears on today's price charts.
ELLIOTT WAVE THEORY - My interpretation of Elliott wave theory implies that stock market has begun its 5th Major Wave lower of which the current impulse wave lower counts as wave 1, time wise this targets a decline for another 5 months which confirms the original forecast for a July 2009 low. However price wise EW targets much lower prices than the original target of 6,600...
Is Major Decline in Gold and Silver Stocks Underway? — The Market Oracle
Before analyzing the recent developments in the mining stocks, let's see what's happening in the general stock market (charts courtesy by http://stockcharts.com.). In the long-term S&P 500 Index chart (related ETF: SPY), we see that prices moved lower …
Forex Trading - An Introduction to Technical Analysis & Chart Patterns
eBooks (Wells Gray Press)