From the course: GenAIOps Foundations
Join today to access over 24,800 courses taught by industry experts.
Memory and embedding management
From the course: GenAIOps Foundations
Memory and embedding management
- [Person] Next, let's discuss memory and embeddings, and how they should be managed under the GenAI apps framework. The GenAI application stack may have classical data stores to store business data. In addition, for GenAI purposes, it also needs to store short-term and long-term memory. RAG applications may require storing and managing embeddings. What memory do we use for GenAI, and what are the key operations considerations for them? Let's talk about short-term memory. Short-term memory is used to store the conversation history between the user and the model. It can be tracked for a single prompt, capturing all individual steps taken to execute the prompt. It can also track all prompts and responses for a given user session in a chat-bot. Typically, for memory, caches are used to store such data. It can also be stored locally within the application memory, or can use external caches like... (indistinct) When building GenAI applications, consideration should be given to the required…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.