Arize AI, a leader in machine learning observability, unveiled broader Arize platform support for generative AI and a unique open-source observability library for evaluating large language models (LLMs) at its Arize: Observe 2023 Summit.
The launch comes at a critical time for the future of AI. Generative AI is fueling a technical renaissance, with models like GPT-4 showing sparks of artificial general intelligence and new breakthroughs and uses cases every day. On the other hand, most leading large language models are black boxes, which have known problems with hallucinations and problematic distortions.
Available today, Arize Phoenix is the first open-source observability library purpose-built to help data scientists evaluate results from LLMs such as OpenAI’s GPT-4, Google’s Bard, Anthropic’s Claude, and others. Using Phoenix, data scientists can visualize complex LLM decisions, monitor LLMs when they are producing incorrect or misleading results, and isolate fixes to improve results.
“A major obstacle to the use of LLMs and generative agents in production lies in the lack of observability of these systems,” said Harrison Chase, co-founder of LangChain. “With Phoenix, Arize offers an open-source method for visualizing complex LLM decisions.”
“Phoenix is a much-appreciated advance in model observability and production,” said Christopher Brown, CEO and co-founder of AI-focused consulting firm Decision Patterns and a former UC Berkeley professor of computer science. “Integrating observability utilities directly into the development process not only saves time but encourages the model development and production teams to actively think about the use of the model and ongoing improvements before releasing it to production. This is a great win for model lifecycle management.”
“Despite calls to halt AI development, the reality is that innovation will continue to accelerate,” said Jason Lopatecki, CEO and co-founder of Arize AI. “Phoenix is the first software designed to help data scientists understand how GPT-4 and LLMs are thinking, monitor their responses, and fix the inevitable problems as they arise.”
Phoenix is instantiated in a Jupyter notebook through a simple import call and is designed to run interactively on Pandas data frames. The tool works seamlessly with unstructured text and images, with embedding and latent structure analysis designed as the core foundation of the toolset.
