Repository navigation
Conversation
…e-9.0 COPY options DISCARD, LISTEN and UNLISTEN failed, SHOW TIME ZONE, SHOW TRANSACTION ISOLATION LEVEL and SHOW SESSION AUTHORIZATION were unsupported statements, and COPY rejected the option syntax from before 9.0 that PostgreSQL 18 still accepts, such as COPY t TO STDOUT CSV HEADER, and COPY BINARY t. DISCARD parses to DiscardStatement with a Target enum, and LISTEN and UNLISTEN to ListenStatement next to NotifyStatement; LISTEN and UNLISTEN become non-reserved keywords. SET, RESET and SHOW read TIME ZONE and PostgreSQL's TRANSACTION ISOLATION LEVEL and SESSION AUTHORIZATION through one ConfigurationParameterName() rule: the SHOW phrases parse to ShowStatement and RESET takes them too. CopyStatement keeps the options in parentheses in getOptions() and adds getLegacyOptions(), a typed CopyLegacyOption per option in source order, and isBinary(). The forms are read with Dialect.POSTGRESQL and without a dialect; other dialects are unchanged. The statements are wired into the deparser, visitors, table finder, validation (discard, listen, and show for PostgreSQL) and feature analysis. Signed-off-by: minleejae <mmj9808@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
PostgreSQL session scripts, connection-pool reset hooks and old dumps use statements that fail on
master(12548b9da) withDialect.POSTGRESQLand without a dialect, for example:DISCARD ALL,DISCARD PLANS,DISCARD SEQUENCES,DISCARD TEMP(the reset query of connection pools such as PgBouncer)LISTEN ch,UNLISTEN ch,UNLISTEN *COPY t TO STDOUT CSV HEADER,COPY t FROM STDIN WITH CSV HEADER DELIMITER ';' NULL '',COPY t TO STDOUT CSV FORCE QUOTE *,COPY BINARY t TO STDOUT: the COPY option syntax from before 9.0, which PostgreSQL 18 still accepts and older tools and scripts still writeSHOW TIME ZONE,SHOW TRANSACTION ISOLATION LEVELandSHOW SESSION AUTHORIZATIONparse, but only toUnsupportedStatement, andRESET TRANSACTION ISOLATION LEVELandRESET SESSION AUTHORIZATIONfail.PostgreSQL 18 references: DISCARD, LISTEN, UNLISTEN, SHOW, RESET, COPY (section "Compatibility" for the older syntax), and
gram.y(DiscardStmt,ListenStmt,UnlistenStmt,VariableShowStmt,reset_rest,CopyStmt,copy_opt_list,copy_opt_item,opt_binary).Current design (As-is)
DISCARD,LISTENorUNLISTEN.DISCARDis a non-reserved keyword used only by MySQL'sALTER TABLE … DISCARD [PARTITION …] TABLESPACEandALTER USER … DISCARD OLD PASSWORD;LISTENandUNLISTENare plain identifiers.NotifyStatementmodelsNOTIFY channel [, 'payload'].Show()reads the typed MySQL forms and otherwise captures the rest of the statement: one word becomesShowStatement(name)(SHOW search_path,SHOW ALL), several words anUnsupportedStatement, soSHOW TIME ZONEis unsupported.RESETreadsTIME ZONEas the parameterTime Zone, a name orALL;SETreadsTIME ZONEwith its own copy of the same rule. Both match any date/time keyword beforeZONE, soRESET DATE ZONEandSET DATE ZONE 'UTC'are rewritten toRESET Time ZoneandSET Time Zone 'UTC'.CopyStatement(DuckDB, and PostgreSQL since feat: support PostgreSQL COPY WITH options #2658) readsCOPY {table [(columns)] | (query)} {FROM | TO} path [WITH] [(options)];getOptions()keeps each option in parentheses as written. Options without parentheses andCOPY BINARY tablefail.Target design (To-be)
The new forms are read with
Dialect.POSTGRESQLand without a dialect; other dialects are unchanged.DISCARD {ALL | PLANS | SEQUENCES | TEMP | TEMPORARY}parses to the newDiscardStatementwith the enumDiscardStatement.Target;TEMPandTEMPORARYstay separate constants so that the statement prints as written (asTemporaryOptiondoes). Without a dialect a target must followDISCARD, soDISCARD foostays an unsupported statement.LISTEN channelandUNLISTEN {channel | *}parse to the newListenStatementinstatement.notify, next toNotifyStatement:getAction()(LISTEN,UNLISTEN),getChannel()(null for*) and the derivedisAllChannels().LISTENandUNLISTENbecome non-reserved keywords, likeUNLOCK(feat: parse MySQL LOCK TABLES, UNLOCK TABLES and the backup lock statements #2888), and remain usable as names. As ingram.y(ColId), a channel cannot be a qualified name, a string or an unquoted word that PostgreSQL reserves or keeps for types and functions (LISTEN all,UNLISTEN binary);"all"is a channel. CREATE RULE keeps taking onlyNOTIFYas an action.SHOW/RESETparameter names: one production,ConfigurationParameterName(), readsTIME ZONE,TRANSACTION ISOLATION LEVEL,SESSION AUTHORIZATIONor a name.RESETuses it,SHOWuses it when one of the three phrases makes up the whole statement, andSETuses it forTIME ZONE(its other phrases set a transaction mode or the session user, not a parameter).SHOW TIME ZONEparses toShowStatementwithgetName()Time Zone(the nameSETandRESETalready give it), and the other two keep their words.TIME ZONEnow requires the wordTIME, soRESET DATE ZONEandSET DATE ZONE 'UTC'are rejected instead of rewritten, as PostgreSQL rejects them.SHOW TIME ZONE xandSHOW TRANSACTION ISOLATIONkeep the genericUnsupportedStatement.CopyStatementgainsgetLegacyOptions(): the options of the syntax from before 9.0, in source order, each aCopyLegacyOptionwith aKind(BINARY,FREEZE,DELIMITER,NULL,CSV,HEADER,QUOTE,ESCAPE,FORCE_QUOTE,FORCE_NOT_NULL,FORCE_NULL,ENCODING), the string (getValue(), aStringValue) andisAsKeyword(), or the columns of aFORCEoption (getColumns(),isAllColumns()for*). Ascopy_opt_list, the options follow the file name and the optionalWITHin any order, without commas, also afterCOPY (query) TO;ENCODINGtakes noAS, and aFORCEcolumn is an unqualified name.isBinary()forCOPY BINARY table. A word afterBINARYis needed, soCOPY binary TO …still copies a table namedbinary.getOptions()(List<String>, DuckDB's free-form options included), unchanged. A statement takes either form;COPY t TO STDOUT CSV (FORMAT csv)andCOPY t TO STDOUT (FORMAT csv) CSVare rejected, as in PostgreSQL.FORCE INSTALLon the next line still starts the next statement;FORCEcontinues the options only beforeQUOTE,NOTorNULL.StatementVisitordefault methods andStatementVisitorAdapter;StatementDeParserrenders through the models'appendTo, shared withtoString();TablesNamesFinderreports no table forDISCARDandLISTEN/UNLISTENand the copied table forCOPY; validation featuresdiscardandlisten, enabled withshowin the PostgreSQL capabilities (SHOWwas missing there); feature analysis reportsMODIFIES_SESSIONforDISCARD,LISTENandUNLISTEN, and a possibleMODIFIES_SCHEMAforDISCARD ALL,TEMPandTEMPORARY, which drop the temporary tables of the session;isStatementStartAhead()listsDISCARD,LISTENandUNLISTEN. The new objects carry AST nodes spanning their source (fix: give every parsed object a node spanning its source #2870). The usage guide, README and index list the statements.COPY … TO PROGRAM 'cmd',COPY … FROM … WHERE conditionand the pre-7.3[USING] DELIMITERS 'x'beforeWITH, which are not option syntax; dotted parameter names inRESET(RESET a.b).Validation
DiscardStatementTest19,ListenStatementTest32, 54 inPostgreSqlCopyTest, 12 inShowStatementTest, 13 inResetStatementTestandPostgreSqlSessionStatementValidatorTest3;ASTNodeCoverageTestwalks the new forms. They fail onmaster.toString(),StatementDeParserand reparse withDialect.POSTGRESQLand without a dialect, plus the AST fields (target, action, channel, parameter name, option kinds, strings,AS,FORCEcolumns and*,BINARY).DISCARD,DISCARD TABLES,DISCARD ALL PLANS;LISTEN,LISTEN *,LISTEN a.b,LISTEN 'ch',LISTEN all,UNLISTEN a, b;RESET TRANSACTION ISOLATION,RESET DATE ZONE;COPY t TO STDOUT CSV (FORMAT csv),… (FORMAT csv) CSV,ENCODING AS 'x',DELIMITERwithout a string,NULL 1,HEADER true,CSV, HEADER,… CSV HEADER WITH,FORCE QUOTEwithout columns,FORCE QUOTE (a),FORCE QUOTE t.a,FORCE QUOTE a, *,COPY BINARY (SELECT 1) TO STDOUT, andLISTEN chas a CREATE RULE action.DISCARD,LISTEN,UNLISTEN, the PostgreSQLRESETphrases and the COPY forms, and keepSHOW TIME ZONEanUnsupportedStatement;ALTER TABLE t DISCARD TABLESPACE, DuckDB'sCOPY binary TO 'out.csv' (FORMAT CSV)andFORCE INSTALLafter aCOPY, the parenthesized COPY options,SHOW name,SET TIME ZONE, andlisten,unlisten,discardas names.CheckpointStatementTest,ClusterStatementTest,VacuumStatementTest,ReindexTest,SecurityLabelTest) usedLISTEN chas an example of an unsupported statement; they useIMPORT FOREIGN SCHEMA s FROM SERVER v INTO tnow.masterrejects. Some fail only when run, for exampleCOPY FORCE_QUOTE requires CSV modeor a missing file.COPY BINARY s.t TO STDOUT CSV FORCE QUOTE afails with SQLSTATE 42601 "conflicting or redundant options". The server raises this while it processes the options, not in its grammar, as it does forCOPY t TO STDOUT CSV CSV, so the parser accepts it. It reports a syntax error for all 57 statements the tests reject, including the threeRESET/SET … DATE ZONEforms thatmasterrewrote toTime Zone. Malformed SHOW phrases such asSHOW TIME ZONE xstayUnsupportedStatement, as onmaster. MySQL 8.4.11 rejects every new form, andDialect.MYSQLrejects them too.spotlessApply cleanTest check jar: 12,922 tests, 12,897 passed, 25 skipped, zero failures/errors. The grammar choice conflict check, Checkstyle, PMD and SpotBugs pass.master(12548b9da) on a corpus of 188 MySQL 8.4 and 166 PostgreSQL 18 statements other than DDL/DCL, each accepted by the server, with the MySQL or PostgreSQL dialect and without a dialect:Dialect.POSTGRESQLand without a dialect:COPY t TO STDOUT CSV HEADER,SHOW TIME ZONE,SHOW TRANSACTION ISOLATION LEVEL(both wereUnsupportedStatement),DISCARD ALL,DISCARD PLANS,LISTEN chandUNLISTEN *.master(12548b9da) on a corpus of 635 MySQL 8.4 and PostgreSQL 18 DDL/DCL statements, each accepted by the server, plus variants, scripts and the keyword sweeps: 0 regressed, 0 changed.AI assistance: prepared with Claude Code.