NTD in AI: Long Short Term Memory (LSTM)

Non-technical definitions in AI

Long Short-Term-Memory (LSTM) is a type of Recurrent Neural Network (RNN) which uses information from outputs a few steps (epochs) before the current layer as part of the output calculations.

Long short-term-memory is so called because it is not long term memory, rather it is memory that is longer than short term memory.

For neural networks, short term memory is information fed from one output layer to the input of the next layer in a single epoch. If you run an RNN for 20 iterations or so it will forget its original state.

For LSTM, some information a few epochs old is calculated and preserved before it is used in the Activation Function. This preserved information (memory) is then fed into the next layer as part of the input for that layer’s Activation Function.

Note that because the function that determines the memory has mathematical operators, if you go too far back, the information eventually becomes noise.

LSTM is used in natural language processing (NLP), i.e. speech or text recognition, machine language translation, sentiment analysis from text or speech, time series forecasting (e.g. stock movement) etc.

Machine learning is a technical subject and the use of technical terms by engineers have the potential of coming between clear communication with non-engineers, especially in the business setting. In spare moments I started to put together simple, non-technical definitions of nouns and verbs used in the field of machine learning as a kind of Rosetta Stone for non-engineers.This is a work-in-progress which I may collect into a book one day. This is one of those definitions.

NTD in AI: Long Short Term Memory (LSTM)

Other non-technical definitions: