How to Use Spring AI with OpenAI Tutorial: A Step-by-Step Guide

Bringing LLMs to the JVM

For a long time, the AI revolution felt like it was happening exclusively in Python. If you wanted to integrate Large Language Models (LLMs), you usually had to build a side-car Python service or rely on raw HTTP clients to call APIs. However, with the release of Spring AI, the game has changed. In this how to use spring ai with openai tutorial, I’ll show you how to integrate GPT-4 directly into your Java applications using a portable, standardized API.

I’ve spent the last few weeks testing Spring AI in a production-like environment, and what impressed me most is how it abstracts the underlying AI provider. This means you can start with OpenAI and potentially switch to Azure AI or Mistral without rewriting your entire business logic. If you’re already leveraging java virtual threads in spring boot to handle high concurrency, adding AI capabilities via Spring AI is the logical next step for building scalable, intelligent agents.

Prerequisites

Before we dive into the code, ensure you have the following ready:

JDK 17 or higher: Spring AI requires a modern JVM.
Spring Boot 3.2+: The latest versions provide the best compatibility.
OpenAI API Key: You’ll need an active account and a generated API key from the OpenAI platform.
Maven or Gradle: I’ll be using Maven for this tutorial.

Step 1: Adding Spring AI Dependencies

First, we need to add the Spring AI Bill of Materials (BOM) to our pom.xml. This ensures all AI-related dependencies use compatible versions. I recommend using the Spring Milestone repository since Spring AI is evolving rapidly.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.0-M1</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
    </dependency>
</dependencies>

Step 2: Configuring the OpenAI API Key

Never hardcode your API keys in your source code. I always use environment variables or a application.properties file that is excluded from Git. Add the following line to your configuration:

spring.ai.openai.api-key=${OPENAI_API_KEY}

To set this in your terminal, use: export OPENAI_API_KEY='your_key_here'. This ensures that your credentials remain secure even if your code is pushed to a public repository.

Step 3: Implementing the AI Service

Now, let’s create a simple service to handle the interaction with OpenAI. The core of Spring AI is the ChatClient. In my experience, creating a dedicated service wrapper makes it much easier to implement spring boot hexagonal architecture patterns, as you can isolate the AI provider from your core domain.

@Service
public class AiChatService {

    private final ChatClient chatClient;

    public AiChatService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String getAiResponse(String userPrompt) {
        return chatClient.prompt()
                .user(userPrompt)
                .call()
                .content();
    }
}

As shown in the image below, the interaction flows from the user request through the ChatClient, which handles the JSON serialization and HTTP transport to OpenAI’s servers automatically.

Technical flow diagram showing the request path from Spring Boot ChatClient to OpenAI API

Step 4: Creating the REST Controller

To make this accessible via an API, we’ll create a simple controller. This allows you to test your implementation using Postman or Curl.

@RestController
@RequestMapping("/ai")
public class AiController {

    private final AiChatService aiChatService;

    public AiController(AiChatService aiChatService) {
        this.aiChatService = aiChatService;
    }

    @GetMapping("/generate")
    public String generate(@RequestParam(value = "message") String message) {
        return aiChatService.getAiResponse(message);
    }
}

Pro Tips for Production AI

Prompt Templates: Don’t just pass raw user input. Use PromptTemplate to provide context (e.g., “You are a senior Java architect. Please answer the following: {userPrompt}”).
Streaming Responses: For a better UX, use .stream() instead of .call(). This allows you to push tokens to the frontend via Server-Sent Events (SSE).
Error Handling: Wrap your calls in a try-catch block to handle RateLimitExceededException or InvalidApiKeyException.

Troubleshooting Common Issues

If you encounter a 401 Unauthorized error, double-check that your environment variable is actually loaded. I’ve often seen developers set the variable in one terminal window and run the app in another. Use System.getenv("OPENAI_API_KEY") in a temporary print statement to verify.

If you experience slow response times, check if you are using gpt-4o or gpt-3.5-turbo. You can configure the model in your properties file: spring.ai.openai.chat.options.model=gpt-4o.

What’s Next?

Once you’ve mastered this how to use spring ai with openai tutorial, the next step is implementing RAG (Retrieval-Augmented Generation). This allows you to feed your own PDFs or database records into the prompt so the AI can answer questions about your specific business data. You can also explore vector databases like PGVector or Pinecone, which integrate seamlessly with Spring AI.

Want to optimize your AI backend? I recommend checking out my guide on using virtual threads to ensure your application doesn’t block while waiting for the LLM to respond.