Gateway to the cloud? Try, access point to AI! That’s the story from API kingmaker Kong, the cloud integration vendor with a storied history. The company just announced a suite of open-source plugins that transform Kong Gateway into a platform for building and deploying multi-LLM applications, simplifying the integration of AI models and accelerating the development process.
One of the key features of the AI Gateway is integration with multiple Large Language Models (LLMs). The “ai-proxy” plugin offers native support for industry leaders like OpenAI, Azure AI, Cohere, Anthropic, Mistral, and LLAMA. This standardized interface allows developers to switch between models, facilitating experimentation and finding the best fit for their specific needs.
Centralized AI credential management ensures secure storage of sensitive information within Kong Gateway, eliminating the need for storing credentials within applications. This simplifies credential rotation and updates, enhancing security and compliance. And it also creates a significant control point for helping organizations achieve AI Governance.