Repository navigation
build(python): migrate pydevlake and plugins to Pydantic 2 - #9352
Merged
Merged
Conversation
Move pydevlake from Pydantic 1.10 / SQLModel 0.0.8 / SQLAlchemy 1.4 to
Pydantic 2.14 / SQLModel 0.0.38 / SQLAlchemy 2.0 and port azuredevops,
glpi, redmine, zephyr and the e2e fakeplugin.
- Optional fields get explicit None defaults (required in Pydantic 2).
- Config classes, validators and __fields__ use the Pydantic 2 APIs.
- ToolTable keeps v1 behavior for numbers in str fields
(coerce_numbers_to_str).
- autoextract validates the extracted values so they are coerced.
- glpi reduces expanded dropdown values ('Name (id)') to the id before
extraction; the names are still read from the raw data.
- Regenerate the Poetry locks; Dependabot no longer ignores Pydantic
majors, SQLModel stays ignored (0.0.48 rejects naive datetimes).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
klesh
approved these changes
Nov 10, 2026
klesh
left a comment
Contributor
There was a problem hiding this comment.
LGTM
Thanks for your contribution.
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pr-type/bug-fix,pr-type/feature-development, etc.(I can't set labels myself; suggested:
pr-type/refactor,component/plugins)Summary
Moves the Python plugin framework from Pydantic 1 to Pydantic 2, as agreed in
#9092. Pydantic 1 does not support Python 3.14, so this is the first step
before the Python runtime (3.11, EOL 2027-10) can be upgraded.
Ported:
pydevlake,azuredevops,glpi,redmine,zephyrand the e2efakeplugin. The three Poetry locks only change the Pydantic stack (plus itsnew dependencies
pydantic-core,annotated-types,typing-inspection;sqlalchemy2-stubsis dropped).pydevlake
model_config/ConfigDict,field_validator,model_fields,model_dump_json,RootModel,Literalinstead ofField(const=True).Optional[...]fields get an explicit= Nonedefault (in Pydantic 2 theyare required otherwise). This accounts for most of the diff in the domain
layer and plugin models/migrations.
sa_column=Column(...)→sa_type=...(+sa_column_kwargsfor thecreated_at/updated_atdefaults), so columns are not shared betweeninherited tables.
ToolTablesetscoerce_numbers_to_str=Trueto keep the Pydantic 1behavior for APIs that return ids as numbers.
autoextractusesmodel_validate: SQLModel table models skip validationin
__init__, so raw strings would no longer be coerced todatetime/enums.RawModelkeeps its column types (data/inputbinary,urlTEXT), soexisting raw tables stay compatible. Two tests wrote
strintodata;they now encode to bytes like the collector does (SQLAlchemy 2 rejects
strfor binary columns).message.py:SerializeAsAnyforRemoteScope.data, otherwise Pydantic 2serializes the base type and drops plugin-specific scope fields.
model_info.py: readsjson_schema_extra/ annotations instead offield.field_info.extra/field.required.so both sides have the same field types.
Plugins
glpi: GLPI expands dropdowns to'Name (id)'; these values are reduced tothe id before validation of the integer columns. The names are still read
from the raw data for enrichment, so the output does not change.
redmine,zephyr:class Config→model_config.fakeplugin: ids as strings,Optionaldefaults.Dependabot
pydanticmajor-version ignore is removed.sqlmodelignore stays: 0.0.48 rejects naive datetimes (pydevlake usesdatetime.now()/utcnow()) and pulls in SQLAlchemy 2.1. That is a separatefollow-up.
Does this close any open issues?
Part of #9092 (Pydantic step; Python runtime upgrade follows separately).
Screenshots
N/A
Other Information
Tests
cd backend/python && sh ./run_tests.sh— passed:azuredevops 8 passed / 1 skipped, glpi 72 passed / 1 skipped,
redmine 31 passed / 1 skipped, zephyr 26 passed, pydevlake 13 passed.
make unit-testandmake e2e-test(MySQL and PostgreSQL, includes thePython
fakeplugin) — passed in the fork CI: run 42621392320 (16/16 jobs)Not in this PR
backend/Dockerfileand the lake-builder image.