Kong Introduces Six New AI Plugins in Kong Gateway 3.6

20 February, 2024

No Comments

Kong has introduced a collection of six open-source AI plugins for its Kong Gateway 3.6 that offer integration with multiple LLMs and significantly improve the accessibility of AI technologies for developers and platform teams.

Kong Gateway 3.6 aims to increase developer productivity by easing the process of integrating LLMs into various products. The enhanced version provides a consistent and controllable environment for AI requests across organizations.

“Today marks a significant milestone in our journey towards democratizing AI for developers and enterprises worldwide. By open-sourcing this suite of innovative AI capabilities, including no-code AI plugins, we’re removing the barriers to AI adoption and making it possible for developers to leverage multiple LLMs effortlessly and ship AI-powered applications faster. At the same time, we’re providing governance and visibility to all the AI traffic that is being generated by an organization,“ said Marco Palladino, CTO and co-founder of Kong Inc.

The “ai-proxy” plugin enables the inclusion of multiple LLMs, such as those offered by OpenAI, Azure AI and others, providing a unified interface that allows developers to switch between models without changing their application code.

The “ai-proxy” plugin also enables the collection of detailed Layer 7 AI metrics, such as tracking the number of request and response tokens, as well as usage data across different providers and LLM models.

In addition, with the introduction of codeless AI integration plugins, most notably the “ai-request-transformer” and “ai-response-transformer”, users can embed AI capabilities directly into API requests and responses.

The “ai-prompt-template” plugin allows for centralized management of prompt templates, enabling quick updates and conformance to approved templates without the need to modify application code.

The “ai-prompt-decorator” and “ai-prompt-guard” plugins, in turn, provide mechanisms to ensure that AI prompts are both compliant with organizational policies and protected from unauthorized or sensitive content.