Skip to content

Add GPT-5.5 and GPT-5.6 Tiktoken support#7661

Merged
tarekgh merged 1 commit into
mainfrom
copilot/add-gpt-5-5-gpt-5-6-support
Jul 16, 2026
Merged

Add GPT-5.5 and GPT-5.6 Tiktoken support#7661
tarekgh merged 1 commit into
mainfrom
copilot/add-gpt-5-5-gpt-5-6-support

Conversation

Copilot AI commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Adds Tiktoken model resolution for GPT-5.5 and GPT-5.6 using the o200k_base encoding.

Changes

  • Support exact model names: gpt-5.5, gpt-5.6
  • Support prefixed variants such as gpt-5.5-mini and gpt-5.6-nano
  • Extend model creation coverage for both families
TiktokenTokenizer.CreateForModel("gpt-5.6");

Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds GPT-5.5 and GPT-5.6 model-name resolution to the ML.NET TiktokenTokenizer so CreateForModel(...) maps these models (and their -<variant> suffix forms) to the o200k_base encoding, with test coverage updates.

Changes:

  • Extend TiktokenTokenizer model-to-encoding mappings to include gpt-5.5 / gpt-5.6 and gpt-5.5-* / gpt-5.6-* prefixes.
  • Add test cases validating creation and supported-name handling for gpt-5.5 / gpt-5.6 and representative suffix variants.
Show a summary per file
File Description
src/Microsoft.ML.Tokenizers/Model/TiktokenTokenizer.cs Adds exact and prefix model mappings for GPT-5.5/GPT-5.6 to O200kBase.
test/Microsoft.ML.Tokenizers.Tests/TiktokenTests.cs Extends supported-model test matrices to include GPT-5.5/GPT-5.6 (and variant examples).

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@tarekgh

tarekgh commented Jul 16, 2026

Copy link
Copy Markdown
Member

/ba-g opened the issue #7662 to track unrelated failures.

@tarekgh tarekgh merged commit 11ed3fe into main Jul 16, 2026
5 of 27 checks passed
@tarekgh

tarekgh commented Jul 16, 2026

Copy link
Copy Markdown
Member

Thanks @stephentoub!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants