test(ai): Update LiveAPI model names in integration tests #9420
Conversation
|
Summary of ChangesHello @hsubox76, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the LiveAPI integration tests to accommodate recent changes in AI model behavior and requirements. It specifically addresses the need to use new model names, enforce a particular region for Vertex AI, and adapt test expectations from text-based responses to audio-based responses, including handling cases where the model might indicate a 'thinking' state. These changes ensure the tests accurately reflect the current operational characteristics of the LiveAPI. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates constants and integration tests for the Live API. The constants.ts file was modified to introduce liveBackends specifically for Vertex AI in us-central1 and Google AI, update liveModelNames to newer model identifiers, and ensure liveTestConfigs utilize these new backends. In live.test.ts, the nextTurnText helper function was refactored and renamed to nextTurnData, now returning an object that includes text, hasAudioData, and hasThinking flags. This change aligns with the updated textLiveGenerationConfig which now expects ResponseModality.AUDIO. As a result, many test assertions were updated to check for the presence of audio data rather than specific text content. Review comments noted that these updated assertions weaken the tests by no longer validating the model's actual response content, suggesting a need for transcription or an explanation. Additionally, a misleading error message in the nextTurnData function was identified, and a typo in the finalresponseData variable name was pointed out.
Size Report 1Affected ProductsNo changes between base commit (5511b4f) and merge commit (0e6f540).Test Logs |
Size Analysis Report 1Affected ProductsNo changes between base commit (5511b4f) and merge commit (0e6f540).Test Logs |
2 participants
This also required updating the LiveAPI integration tests, as the new models don't allow
ResponseModality.TEXTand the Vertex model currently requires regionus-central1.The Developer API handles multiple chunks strangely and often returns only a thinking part.