OPENJPA-2940 - Jakarta Persistence 3.2 #142
Conversation
* Updated tentative version to 4.2.0-SNAPSHOT * Updated java version to 17
* Updating project to exclude almost all java.security deprecated calls * Passes default profile tests, but fails with postgres * Fixed some non-deterministic tests that fail with postgresql
* Tested and passed XML support using postgresql-17 as target db
* Replacing string number constructors * Removing dangling SecurityContext references
* removed TestSecurityContext because it is terminally deprecated since 17 and already removed in current JDK versions * updated h2-2 test profile jdbc url to remove strict definition * updated openjpa-slice and openjpa-xmlstore pom system variables definitions * updated GH actions workflows to use test-h2-2 profiles
* Project passes tests on derby, h2-2, postgres:latest, mysql:lts, mariadb:lts
* Updated dependency version * Added API new methods to API implementation classes with methods that throw UnsupportedOperationException, except for four methods in EntityManagerImpl that required proper implementations to pass tests * Project is still passing tests on derby and postgresql, at least
* Added XML JPA 3.2 schema and definitions * Added configuration support by 3.2 version * Added SchemaManager impl and corresponding methods in BrokerFactory interface * Added concrete working (not for h2-2) implementation of SchemaManager methods for JDBCBrokerFactory * Added concrete working impl for EMF#getName()
* Reverting unnecessary changes * Fixing broken map synchronization
* Changing signature of BrokerFactory API on schema dealing validate method * Adding test to check if validate operation throws exception when it fails * Changing GH CI workflow to allow usage of both self-hosted and GH hosted runners * Tested on derby, h2-2, postgresql:latest, mysql:lts, mariadb:lts
* Implementing emf creation passing PersistenceConfiguration
* Removing unused import in BrokerImpl * Implemented new PersistenceUnitUtil load methods
* Moved PUU loading tests to test unit already present * Updated test unit to junit 4.x format
…tSchemaGeneration The static _droppedTables set was unconditionally tracking tables dropped by schema gen scripts, preventing buildSchema/add from re-creating them. This caused cross-test contamination in unit tests where separate EMFs share the same JVM. Gate the tracking on SpecCompliantSchemaGeneration (TCK mode only) and remove redundant mapSchemaGenerationToSynchronize- Mappings call in 3-arg synchronizeMappings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…Generation The JPA spec default FK naming convention for @mapsid (relationship field + "_" + referenced PK column) breaks backward compatibility for existing databases using OpenJPA's original naming. Gate the renaming on SpecCompliantSchemaGeneration so existing users are not affected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… 3.2) JPA 3.2 requires getResultList() to return a mutable List. OpenJPA now wraps ResultList in ArrayList, so iterators remain valid after query close and ResultList-specific APIs are no longer directly accessible. - TestQueryResults: iterators stay valid after close (expected behavior) - TestExternalizedParameter: verify queries execute without relying on ResultList.getUserObject() internal API Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ameters When stored procedure parameters are registered by name only, getOutputParameterValue(int) should accept 1-based positions corresponding to the declaration order, not reject them as invalid. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… attributes getId() and getDeclaredId() used pick() which returns only the first ID attribute. For IdClass entities with multiple IDs, this fails when the first attribute doesn't match the requested type. Iterate all ID attributes and match by type instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The try/catch for IOException in parseResources() was swallowing XML validation errors (SAXException wrapped in IOException), causing MissingResourceException instead of the expected GeneralException for invalid persistence.xml files. Re-throw when the cause is SAXException while still skipping actual I/O errors (corrupt jars). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Filter static enhancer fields in determineImplicitAccessType() so callback-only @MappedSuperclass correctly returns EMPTY access, preventing superclass compat check from overriding subclass PROPERTY - Update TestBiDirectionalJoinTable for JPA 3.2 spec 4.10: bulk DELETE does not cascade to related entities - Fix CompareByExample to recurse into ManagedType attributes (both associations and embeddables) after isAssociation() was corrected to exclude embeddable types Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update TestFieldAccessWithNonStandardAccessors: getdescription() is a valid property accessor per JPA 3.2 TCK, so description IS persistent - Remove SpecCompliantSchemaGeneration gate from SchemaTool dropped-table tracking; use remove() instead of contains() to prevent cross-test contamination while preserving drop-then-rebuild semantics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…and MetaDataDefaults - Extract addSemicolonDelimited() in JDBCBrokerFactory to consolidate two identical type-list parsing loops - Extract getInstanceFields() in EmbedValueHandler to share non-static field iteration between toDataStoreValue1 and toObjectValue1 - Extract toPropertyName() in PersistenceMetaDataDefaults to replace four copies of getter-to-field name conversion logic Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Regarding the CI failure: will have a look tomorrow - I didnt use the h2 profile (as its done for PRs) locally. |
…sts from TestIdTypeFixes These tests duplicated coverage already provided by TestBigDecimalIdSharedTable and used fractional BigDecimal IDs that fail on H2 due to shared-column type resolution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Hi, @rzo1 ! Great work! I've started to test the PR on different DBs. I got some errors (127) and failures (4) on a docker postgresql-18. I will rebuild everything using C locale to be on the safe side and test on MariaDB (LTS and latest), MySQL (LTS and latest). As soon as I have some results, I'll cycle back. |
Great - I expected failures (of the regular build) against various dbms. Do you run those tests with containers? If so - once you have some more results - I can run that as well locally. |
Yes, running in containers (official image of each db), dropping and recreating openjpa testing database in each run, building and testing openjpa also in container (temurin-21) with C locale. |
|
Yeah - feel free. I am sure, we can get it to work :) |
|
thanks @rzo1 - thats a huge and nice contribution! |
|
After reverting changes on UUID handling, the branch passes on profile test-h2-2 both using in-memory and tcp server mode, preserving UUID field type on database. The test-postgresql profile was tested using the same environment, pointing to a docker pg:17, locale en_US.UTF-8. [ERROR] Tests run: 3488, Failures: 2, Errors: 139, Skipped: 0 |
|
@cristof Thanks fpr the feedback so far. I am going to reproduce this on my system and running the same tests :) Results: 3608 tests, 4 failures, 137 errors w/ PostgreSQL 17 (in the profile) - going to check. Compared to |
…chema drop tracking UUID lob path: Added UUID_OBJ case to the lob branch of getJDBCType() so UUID columns get Types.OTHER instead of Types.BLOB when ImmutableValueHandler sets size=-1. PostgreSQL null char: Override setChar() in PostgresDictionary to convert the Java default char '\0' to SQL NULL, since PostgreSQL rejects 0x00 bytes in CHAR/VARCHAR columns (storeCharsAsNumbers=false). Schema drop tracking: Changed _droppedTables.remove() to contains() in SchemaTool.createTable() so subsequent buildSchema EMFs also skip re-creating tables that were dropped by schema generation scripts.
…UUID and VARCHAR columns Override setUnknown() and setTyped() in PostgresDictionary to bind UUID values as string with Types.OTHER. The PG JDBC driver sends this as PostgreSQL's "unknown" pseudo-type (OID 705), which PostgreSQL can implicitly cast to both native UUID and VARCHAR column types. This is needed because PostgreSQL cannot implicitly cast between its native UUID type and VARCHAR in WHERE comparisons: - setObject(idx, uuid, Types.OTHER) fails on VARCHAR columns - setString(idx, uuid.toString()) fails on native UUID columns - setObject(idx, uuid.toString(), Types.OTHER) works for both Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…iantSchemaGeneration When SpecCompliantSchemaGeneration is true (TCK mode), use contains() so the dropped-table entry persists across multiple buildSchema EMFs within the same schema generation flow. When false (normal mode), use remove() so the entry is consumed and subsequent test classes can re-create the table.
…uotes from numeric identifiers The toDBName() override unconditionally stripped double-quote delimiters from all identifiers to fix case-sensitivity issues. However, identifiers that start with a digit (e.g., column name "1") require quoting in PostgreSQL DDL. Now only strips quotes when the unquoted form starts with a letter. Fixes 5 test classes: TestResultClsAnnotation, TestResultClsXml, TestSimpleXmlEntity, TestXmlOverrideEntity, TestMixedMappingLocation.
…ripping quotes The previous fix only checked the first character of quoted identifiers. Identifiers containing spaces (e.g., "nsc DelSet") were incorrectly unquoted, producing invalid SQL on PostgreSQL. Now validates all characters match PostgreSQL's unquoted identifier rules.
…CompliantSchemaGeneration The unconditional SCHEMA_CASE_LOWER caused convertSchemaCase() to lowercase delimited identifiers during metadata lookups. PostgreSQL preserves case for quoted identifiers, so mixed-case tables like "nsc DelSet" were not found when querying metadata with "nsc delset". This prevented proper DROP TABLE during schema synchronization. Now only forces LOWER in TCK mode (where it prevents "ITEM" vs item duplicate-table issues). In normal mode, uses PRESERVE so quoted identifiers with mixed case work correctly.
Test asserted delimitedCase=LOWER which is now only active in TCK mode. Normal PostgreSQL mode uses PRESERVE, making these assertions invalid. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Working here to find and fix some regressions on pg. Do we have a target db clearance to release? I'm working on pg, h2(2.x), mariadb and mysql. All but h2-2 still failing. |
I think PG17 should be fine with the next commit (at least on my PG17 runs, I only have 2 proxy failures similar to master) - but feel free to just commit direclty on this branch. I didn't look into mariadb and mysql though. |
I think that we should come to a state in which we do not have regressions for existing users (but that is from the side line :) ) |
|
I still get some errors with pg-17 (schemavalidation test is my current bisect focus). mariadb and mysql have much more errors or failures. |
Should be gone with b6fde0b - mariadb and mysql might miss some dictionary updates for the new spec parts. Two things: 1.) Maybe we can share the surefire reports somewhere? Running the > 45min tests all the time is a bit cumbersome or do we have ASF Jenkins CI for it and can configure a run from this PR with multi databases? 2.) I am going to stop working on this branch for different rdbms for now so we avoid doing things multiple times. If you have any numbers or surefire reports for mariadb / mysql / pg17 (the few errors which are left), than feel free to drop them somewhere, so I can take another round :) |
Hi all, @solomax, @cristof, all
This PR builds on @cristof's JPA 3.2 work, completed with the assistance of generative AI (Claude).
It fully passes the JPA 3.2 TCK (at least on PostgreSQL).
Tests run on Java 21, though there are some local failures - similar to what I see on master, likely an OSX or Java 21 issue rather than something introduced here:
The PR is fairly large, so feel free to cherry-pick whatever is useful. I expect there are concerns and things to address before this is merge-ready.
A number of new tests have been added that mirror TCK behavior. Happy to drop those if they're unwanted.
That said, it was a fun three-week experiment :)