diff --git a/changelog b/changelog index b281048f..a0569f1a 100644 --- a/changelog +++ b/changelog @@ -1,3 +1,222 @@ +2023 07 05 - v24.0 + +This major release adds support to migration of SQL Server database to +PostgreSQL. It also fixes several issues reported since past height months +and adds some new features and improvements. + + * Enable the use of ALLOW/EXCLUDE directive with SHOW_* reports and throw + a fatal error if global filters in ALLOW/EXCLUDE are set. + * Add replacement of DBMS_LOCK.SLEEP with pg_sleep + * Split estimate cost details per function/procedure/and package function. + * Add cmin, cmax, ctid to reserved keywords list. + * Add cost for presence of ADD CONSTRAINT in PLSQL code. It needs constraint + name stability. + * Add CLOB_AS_BLOB configuration directive to treat CLOB as BLOB when + exporting data. When enabled Ora2Pg will apply same behavior on CLOB + than BLOB with BLOB_LIMIT setting. This could be useful if you have + large CLOB data. Enabled by default. Thanks to Omar Mebarki for the patch. + * Allow COPY and TABLE type to use the NULLIF construct. Thanks to Luke Davies + for the patch. + * Add new SEQUENCE_VALUES export type to export DDL to set the last values + of sequences from current Oracle database last values like the following + statements: ALTER SEQUENCE departments_seq START WITH 290; + Thanks to sergey grinko for the feature request. + * Add replacement of Oracle variable : varname into PG :'varname'. + * Add SQL Server migration to Ora2Pg. Most of the SQL Server objects are + supported as well as data export. Translation of the TSQL stored + procedures to plpgsql is complicated because of the lack of statement + separator in TSQL but as usual Ora2Pg is doing is best to do as much + work as possible. Migration assessment is also possible with SQL Server + database. There is some dedicated configuration directives added to + ora2Pg.conf. + * Add support to MySQL PARTITION BY KEY() with a translation to HASH + partitioned table using the PK/UK definition of the table or the + columns specified in the KEY() clause. Thanks to Shubham Dabriwala + for the report. + * Make EXPORT_INVALID configuration directive works with TRIGGER export. + Until now disabled triggers were not exported, setting EXPORT_INVALID + to 1 will force the export of disabled triggers. Thanks to chetank-yb + for the report. + * Add support of MySQL generated default value on update. For example: + CREATE TABLE t1 ( + dt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP + ); + Ora2Pg will translate this syntax into a trigger on the table to force + the value of the column on an update event. + Thanks to heysky for the report. + * Add translation of ST_GEOMETRY data type to PostGis geometry datatype. + * Replace ROWNUM in target list with a "row_number() over ()" clause. Thanks + to Rui Pereira for the report. + +New configuration directives: + + * Add configuration directive ST_GEOMETRYTYPE_FUNCTION to be able to set the + function to use to extract the geometry type from a ST_Geometry column. + Default: ST_GeometryType, example it should be set to sde.ST_GeometryType + for ArcSDE. Thanks to Albert88373 for the report. + * Add four new configuration directive to be able to change or prefix the + functions used to extract information from ST_Geometry object and values. + - ST_SRID_FUNCTION: Oracle function to use to extract the srid from + ST_Geometry meta information. Default: ST_SRID, for example it should be + set to sde.st_srid for ArcSDE. + - ST_DIMENSION_FUNCTION: Oracle function to use to extract the dimension + from ST_Geometry meta information. Default: ST_DIMENSION, for example it + should be set to sde.st_dimention for ArcSDE. + - ST_ASBINARY_FUNCTION: Oracle function to used to convert an ST_Geometry + value into WKB format. Default: ST_ASBINARY, for example it should be set + to sde.st_asbinary for ArcSDE. + - ST_ASTEXT_FUNCTION: Oracle function to used to convert an ST_Geometry + value into WKT format. Default: ST_ASTEXT, for example it should be set + to sde.st_astext for ArcSDE. + Thanks to Albert88373 for the report. + * Add INSERT_ON_CONFLICT configuration directive. When enabled this instruct + Ora2Pg to add an ON CONFLICT DO NOTHING clause to all INSERT statements + generated for this type of data export. Thanks to Clemens Rieder for the + feature request. + +Backward compatibility: + + * Change the behavior of CASE_INSENSITIVE_SEARCH to allow the use of a + collation instead of the citext extension. To disable the feature the + value none can be used. If the migration is not MSSQL this feature is + disabled. + * Remove PREFIX_PARTITION configuration directive, it is now replaced by + the RENAME_PARTITION directive. Previous behavior was to construct the + partition name from the table name, the partition name and the sub + partition name if any. The problem is that we often reach the max length + for an object name and this leads to duplicate partition name. Now, when + RENAME_PARTITION is enabled the partition tables will be renamed + following rules: + _part + where "pos" is the partition number. For subpartition this is: + _part_subpart + If this is partition/subpartition default: + _part_default + _part_subpart_default + This change will break backward comaptibilty, if PREFIX_PARTITION is + still set, it will simply enable RENAME_PARTITION. + * Set START value to MINVALUE when a sequence is cycled and that the START + value is upper that MAXVALUE. Thanks to Shane Borden for the report. + +Here is the full list of changes and acknowledgements: + + - Fix MODIFY_STRUCT that was not working with MySQL. Thanks to Code-UV for + the report. + - Fix license string in Makefile.PL. Thanks to RodRaen for the report. + - Do not remove non alphanumeric character in index name. Thanks to gwidt + for the report. + - Reorder trigger event when the update of column is not the last one. Thanks + to tayalarun1 for the report. + - Fix export of MySQL function containing special characters and white spaces + in names. Thanks to Shubham Dabriwala for the report. + - Fix grant export for partitions. Thanks to elexus for the report. + - Add some other transformation for sqlplus/psql scripts. + - Remove comma as possible separator for values in DEFINED_PK, it was + preventing the use of a function with multiple parameters. + - Fix export of geometry tables when PG_SCHEMA is set. + - Add rewriting of some sqlplus settings to psql settings. + - Fix TABLESPACE export for partitioned tables. Thanks to elexus for the + report. + - Fix for Issue #1637. Thanks to Simon Pane for the patch. + - Fix typo in --init_project directories tree generation for sequences + values. + - Fix alias in view target list for function call without alias defined in + MySQL export. Thanks to Shubham Dabriwala for the report. + - Fix Mysql procedure export when a datatype with precision is used in + parameter list. Thanks to Shubham Dabriwala for the report. + - Fix collation on string default values. Thanks to Shubham Dabriwalafor + the report. + - Exclude recycle bin object from ALL_TAB_COLUMNS lookup. Thanks to Dave + Betterton for the report. + - Fix data types translation (TINYINT|SMALLINT|MEDIUMINT|INTEGER|BIGINT|INT) + for MySQL table export. Thanks to Shubham Dabriwala for the report. + - Do not export synonym destination table with table_owner when EXPORT_SCHEMA + is disabled. Thanks to Priyanshi Gupta for the report. + - Fully qualify calls to get_sequence_last_values() when PG_SCHEMA is set. + Thanks to Marius Hope for the report. + - Fix regression on exporting view as table when VIEW_AS_TABLE contains + regexp. Thanks to Neil Bower for the report. + - Fix missing execution of initial command statements at start of TEST_DATA + action and on both side, those applying to source and destination. Thanks + to Petter Jacobsen for the report. + - Fix script to get sequence last value with TEST action. Thanks to franxav06 + for the patch. + - Prepend PERFORM before call to DBMS_OUTPUT.* when USE_ORAFCE is enabled. + - Disable USE_ORAFCE when export type is SHOW_REPORT. + - Extending the enhancement in Pull Request #1621 to the Oracle_FDW user + mapping. Thanks to Simon Pane for the patch. + - Changed prefix string to "DIFF:" in test report. Thanks to Simon Pane for + the patch. + - Fix cases where %ROWCOUNT was not correctly replaced. Thanks to Rui Pereira + for the report. + - Fix parsing of ORACLE_DSN when creating foreign server in COPY mode. Thanks + to Luke Davies for the report. + - Fix for Issue #1622, #1627. Thanks to Simon Pane for the patch. + - Fix index creation with DESC order in COPY action when DROP_INDEXES is + enabled. Thanks to Luke Davies for the report. + - Fix for Issue #1610, #1612, #1617 and #1381. Thanks to Simon Pane for the + patch. + - Fix typo in sqlnet.ora name (was sqlnet.or). Thanks to Martin Nash for the + patch. + - Fix data export, REPLACE_QUERY was not applied. Thanks to Bachev Constantin + for the report. + - Fix call to replace_sys_context(). + - Fix timestamp(n) data type translation. + - Remove use of column GENERATION_EXPRESSION for MySQL version < 5.7.0. Thanks + to Hans Choi for the report. + - Fix conversion of DATE datatype to timestamp(0) instead of timestamp. Thanks + to Akhil Reddy for the report. + - Add NVARCHAR/NCHAR defaut convertion data types to DATA_TYPE configuration + directive in ora2pg.conf comments. Thanks to Akhil Reddy for the report. + - Rename method _get_partitions_type function into _get_partitions_list. + - Fix synonym export when no schema information is available. + - Fix support of REFERENCING clause in triggers. + - Fix partition output file renaming with new RENAME_PARTITION directive. + Thanks to Rahul Barigidad for the report. + - Fix export of the ROWNUM clause when there is a variable. + - Fix sprintf placeholders in geometry queries. + - Fix some others issues with row count report. + - Fix row count with destination schema and when the PostgreSQL table + doesn't exist. + Thanks to bizen-ya for the report. + - Fix tests comparison with the different settings of EXPORT_SCHEMA, + SCHEMA and PG_SCHEMA. Thanks to Marius Hope and bizen-ya for the + report. + - Fix St_AsText() call for MySQL data extraction. + - Add column count comparison for MySQL + - Export multi column partition by list as an expression with concat + operator. Multi column partition by list is not supported by PostgreSQL. + - Fix creation of non existant indexes on partition. Thanks to Shubham + Dabriwala for the report. + - Fix MySQL function export when there is no BEGIN clause. Thanks to + Shubham Dabriwala for the report. + - Fix MySQL export of unsigned numeric. Thanks to Shubham Dabriwala for + the report. + - Fix MySQL output with wrong synthax for JOIN without ON clause. Thanks + to Shubham Dabriwala for the report. + - Fix virtual column export. Thanks to Rafal Hollins for the report. + - Fix index creation on partition with no columns for MySQL export. + Thanks to Shubham Dabriwala for the report. + - Fix export of MySQL auto_increment when PG_INTEGER_TYPE is disabled. + Thanks to Shubham Dabriwala for the report. + - Fix MySQL subpartition export. Thanks to Sanyam Singhal for the report. + - Move any INTO clause in CONNECT BY query to the final SELECT on the + resulting CTE. Thanks to taptarap for the report. + - Fix translation of MySQL curtime() function in default values. Thanks + to Shubham Dabriwala for the report. + - Fix possible "Nested quantifiers in regex" error when exporting package + with package name containing regex special characters. Thanks to durandm70 + for the report. + - Fix documentation about use of unique key for ORACLE_COPY. + - Fix extra comma at end of a CHECK contraint. Thanks to Shubham Dabriwala + for the report. + - Always add DROP TYPE statements with package export even if DROP_IF_EXISTS + is not enabled. Thanks to Rui Pereira for the report. + - Fix default value of simple dot in MySQL export. Thanks to Shubham + Dabriwala for the report. + - Fix regression in data type translation after fix on unsigned numeric type. + Thanks to Shubham Dabriwala for the report. + 2022 10 08 - v23.2 This release fix several issues reported since past height months and diff --git a/doc/ora2pg.3 b/doc/ora2pg.3 index 1d54ead5..89b61b3e 100644 --- a/doc/ora2pg.3 +++ b/doc/ora2pg.3 @@ -133,7 +133,7 @@ .\" ======================================================================== .\" .IX Title "ORA2PG 1" -.TH ORA2PG 1 "2023-06-27" "perl v5.34.0" "User Contributed Perl Documentation" +.TH ORA2PG 1 "2023-06-28" "perl v5.34.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l @@ -3675,7 +3675,7 @@ Any contribution to build a better tool is welcome, you just have to send me your ideas, features request or patches and there will be applied. .SH "LICENSE" .IX Header "LICENSE" -Copyright (c) 2000\-2022 Gilles Darold \- All rights reserved. +Copyright (c) 2000\-2023 Gilles Darold \- All rights reserved. .PP .Vb 4 \& This program is free software: you can redistribute it and/or modify diff --git a/lib/Ora2Pg.pm b/lib/Ora2Pg.pm index 41ba06e0..058d3113 100644 --- a/lib/Ora2Pg.pm +++ b/lib/Ora2Pg.pm @@ -43,7 +43,7 @@ use Encode; #set locale to LC_NUMERIC C setlocale(LC_NUMERIC,"C"); -$VERSION = '23.2'; +$VERSION = '24.0'; $PSQL = $ENV{PLSQL} || 'psql'; $| = 1; diff --git a/lib/Ora2Pg/GEOM.pm b/lib/Ora2Pg/GEOM.pm index 4e43a229..78530b0d 100644 --- a/lib/Ora2Pg/GEOM.pm +++ b/lib/Ora2Pg/GEOM.pm @@ -40,7 +40,7 @@ use vars qw($VERSION); use strict; -$VERSION = '23.2'; +$VERSION = '24.0'; # SDO_ETYPE # Second element of triplet in SDO_ELEM_INFO diff --git a/lib/Ora2Pg/MySQL.pm b/lib/Ora2Pg/MySQL.pm index 5d517786..f25be66a 100644 --- a/lib/Ora2Pg/MySQL.pm +++ b/lib/Ora2Pg/MySQL.pm @@ -10,7 +10,7 @@ use POSIX qw(locale_h); setlocale(LC_NUMERIC,"C"); -$VERSION = '23.2'; +$VERSION = '24.0'; # Some function might be excluded from export and assessment. our @EXCLUDED_FUNCTION = ('SQUIRREL_GET_ERROR_OFFSET'); diff --git a/lib/Ora2Pg/Oracle.pm b/lib/Ora2Pg/Oracle.pm index 7fd10261..66f92d67 100644 --- a/lib/Ora2Pg/Oracle.pm +++ b/lib/Ora2Pg/Oracle.pm @@ -11,7 +11,7 @@ use Encode; #set locale to LC_NUMERIC C setlocale(LC_NUMERIC,"C"); -$VERSION = '23.2'; +$VERSION = '24.0'; # Some function might be excluded from export and assessment. our @EXCLUDED_FUNCTION = ('SQUIRREL_GET_ERROR_OFFSET'); diff --git a/lib/Ora2Pg/PLSQL.pm b/lib/Ora2Pg/PLSQL.pm index 46fef4f0..05f04cea 100644 --- a/lib/Ora2Pg/PLSQL.pm +++ b/lib/Ora2Pg/PLSQL.pm @@ -31,7 +31,7 @@ use POSIX qw(locale_h); setlocale(LC_NUMERIC,"C"); -$VERSION = '23.2'; +$VERSION = '24.0'; #---------------------------------------------------- # Cost scores used when converting PLSQL to PLPGSQL diff --git a/packaging/README b/packaging/README index 8e9c741b..0fb3dff2 100644 --- a/packaging/README +++ b/packaging/README @@ -12,13 +12,13 @@ RPM/ The binary package may be found here: - ~/rpmbuild/RPMS/noarch/ora2pg-23.2-1.noarch.rpm + ~/rpmbuild/RPMS/noarch/ora2pg-24.0-1.noarch.rpm or - /usr/src/redhat/RPMS/i386/ora2pg-23.2-1.noarch.rpm + /usr/src/redhat/RPMS/i386/ora2pg-24.0-1.noarch.rpm To install run: - rpm -i ~/rpmbuild/RPMS/noarch/ora2pg-23.2-1.noarch.rpm + rpm -i ~/rpmbuild/RPMS/noarch/ora2pg-24.0-1.noarch.rpm slackbuild/ @@ -30,11 +30,11 @@ slackbuild/ then take a look at /tmp/build/ to find the Slackware package. To install run the following command: - installpkg /tmp/build/ora2pg-23.2-i386-1gda.tgz + installpkg /tmp/build/ora2pg-24.0-i386-1gda.tgz or - installpkg /tmp/build/ora2pg-23.2-x86_64-1gda.tgz + installpkg /tmp/build/ora2pg-24.0-x86_64-1gda.tgz following the architecture. diff --git a/packaging/debian/ora2pg/DEBIAN/control b/packaging/debian/ora2pg/DEBIAN/control index d1e2ca4a..b17a0594 100644 --- a/packaging/debian/ora2pg/DEBIAN/control +++ b/packaging/debian/ora2pg/DEBIAN/control @@ -1,5 +1,5 @@ Package: ora2pg -Version: 23.2 +Version: 24.0 Priority: optional Architecture: all Essential: no diff --git a/packaging/slackbuild/Ora2Pg.SlackBuild b/packaging/slackbuild/Ora2Pg.SlackBuild index e32e3325..d113b0d5 100644 --- a/packaging/slackbuild/Ora2Pg.SlackBuild +++ b/packaging/slackbuild/Ora2Pg.SlackBuild @@ -12,7 +12,7 @@ ## Fill these variables to your needs ## NAMESRC=${NAMESRC:-ora2pg} -VERSION=${VERSION:-23.2} +VERSION=${VERSION:-24.0} EXT=${EXT:-tar.bz2} NAMEPKG=${NAMEPKG:-ora2pg} PKGEXT=${PKGEXT:-tgz/txz} diff --git a/packaging/slackbuild/Ora2Pg.info b/packaging/slackbuild/Ora2Pg.info index a1d7b787..9c4ee640 100644 --- a/packaging/slackbuild/Ora2Pg.info +++ b/packaging/slackbuild/Ora2Pg.info @@ -1,7 +1,7 @@ PRGNAM="Ora2Pg" -VERSION="23.2" +VERSION="24.0" HOMEPAGE="http://ora2pg.darold.net/" -DOWNLOAD="http://downloads.sourceforge.net/ora2pg/ora2pg-23.2.tar.gz" +DOWNLOAD="http://downloads.sourceforge.net/ora2pg/ora2pg-24.0.tar.gz" MD5SUM="" DOWNLOAD_x86_64="UNTESTED" MD5SUM_x86_64="" diff --git a/scripts/ora2pg b/scripts/ora2pg index 6c4a4a24..5478b6b0 100644 --- a/scripts/ora2pg +++ b/scripts/ora2pg @@ -31,7 +31,7 @@ use POSIX qw(locale_h sys_wait_h _exit); setlocale(LC_NUMERIC, ''); setlocale(LC_ALL, 'C'); -my $VERSION = '23.2'; +my $VERSION = '24.0'; $| = 1; diff --git a/scripts/ora2pg_scanner b/scripts/ora2pg_scanner index 577e51dc..ee1e8b65 100644 --- a/scripts/ora2pg_scanner +++ b/scripts/ora2pg_scanner @@ -26,7 +26,7 @@ use strict; use Getopt::Long qw(:config no_ignore_case bundling); -my $VERSION = '23.2'; +my $VERSION = '24.0'; my @DB_DNS = (); my $OUTDIR = '';