v0.16.2, "The best laid plans"

This is a large-ish bugfix release. - Support, with reservations, binlog_row_image=MINIMAL - parser bug: handle table names that look like floating points - parser bug: fix for entity names that have '.', '\', etc in them - handle UPPERCASE encoding names - support UCS2 (start trying to operate ok on the mysql-test suite) - use ObjectOutputStream.reset to fix memory leaks when buffering to disk
zendesk · Dec 16, 2015 · c2c5223 · c2c5223
1 parent 5610a4f
commit c2c5223
Show file tree

Hide file tree

Showing 7 changed files with 27 additions and 15 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -5,7 +5,7 @@ RUN apt-get update && apt-get install -y openjdk-7-jre curl
 RUN mkdir /app
 WORKDIR /app
 
-RUN curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.1-RC1/maxwell-0.16.1-RC1.tar.gz \
+RUN curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.2-RC1/maxwell-0.16.2-RC1.tar.gz \
   | tar --strip-components=1 -zxvf -
 
 ADD REVISION /

diff --git a/docs/docs/quickstart.md b/docs/docs/quickstart.md
@@ -30,9 +30,9 @@ mysql> GRANT ALL on maxwell.* to 'maxwell'@'localhost';
 You'll need a version 7 of a JVM.
 
 ```
-curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.1-RC1/maxwell-0.16.1-RC1.tar.gz \
+curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.2-RC1/maxwell-0.16.2-RC1.tar.gz \
        | tar zxvf -
-cd maxwell-0.16.1-RC1
+cd maxwell-0.16.2-RC1
 ```
 
 

diff --git a/docs/site/compat/index.html b/docs/site/compat/index.html
@@ -81,6 +81,9 @@
                       <li class="toctree-l3"><a href="#unsupported">Unsupported</a></li>
 
 
+                      <li class="toctree-l3"><a href="#binlog_row_imageminimal">binlog_row_image=MINIMAL</a></li>
+
+
                       <li class="toctree-l3"><a href="#master-recovery">Master recovery</a></li>
 
 
@@ -103,7 +106,16 @@ <h3 id="requirements">Requirements:</h3>
 <h3 id="unsupported">Unsupported</h3>
 <ul>
 <li>Mysql 5.7 is untested with Maxwell, and in particular GTID replication is unsupported as of yet.</li>
-<li><code>binlog_row_image=MINIMAL</code> is not supported and will break Maxwell in a variety of amusing ways.</li>
+</ul>
+<h3 id="binlog_row_imageminimal">binlog_row_image=MINIMAL</h3>
+<p>As of 0.17.0, Maxwell supports binlog_row_image=MINIMAL, but it may not be what you want.  It will differ
+from normal Maxwell operation in that:</p>
+<ul>
+<li>INSERT statements will no longer output a column's default value</li>
+<li>UPDATE statements will be incomplete; Maxwell outputs as much of the row as given in the binlogs,
+  but <code>data</code> will only include what is needed to perform the update (generally, id columns and changed columns).
+  The <code>old</code> section may or may not be included, depending on the nature of the update.</li>
+<li>DELETE statements will be incomplete; generally they will only include the primary key.</li>
 </ul>
 <h3 id="master-recovery">Master recovery</h3>
 <p>Currently Maxwell is not very smart about master recovery or detecting a promoted slave; if it determines

diff --git a/docs/site/mkdocs/search_index.json b/docs/site/mkdocs/search_index.json
@@ -7,7 +7,7 @@
         }, 
         {
             "location": "/quickstart/", 
-            "text": "Row based replication\n\n\nMaxwell can only operate if row-based replication is on.\n\n\n$ vi my.cnf\n\n[mysqld]\nserver-id=1\nlog-bin=master\nbinlog_format=row\n\n\n\n\nGrant permissions\n\n\nmysql\n GRANT ALL on maxwell.* to 'maxwell'@'%' identified by 'XXXXXX';\nmysql\n GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'%';\n\n# or for running maxwell locally:\n\nmysql\n GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'localhost' identified by 'XXXXXX';\nmysql\n GRANT ALL on maxwell.* to 'maxwell'@'localhost';\n\n\n\n\n\nInstall maxwell\n\n\nYou'll need a version 7 of a JVM.\n\n\ncurl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.1-RC1/maxwell-0.16.1-RC1.tar.gz \\\n       | tar zxvf -\ncd maxwell-0.16.1-RC1\n\n\n\n\nRun with stdout producer\n\n\nUseful for smoke-testing the thing.\n\n\nbin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' --producer=stdout\n\n\n\n\nIf all goes well you'll see maxwell replaying your inserts:\n\n\nmysql\n insert into test.maxwell set id = 5, daemon = 'firebus!  firebus!';\nQuery OK, 1 row affected (0.04 sec)\n\n(maxwell)\n{\ntable\n:\nmaxwell\n,\ntype\n:\ninsert\n,\ndata\n:{\nid\n:5,\ndaemon\n:\nfirebus!  firebus!\n},\nts\n: 123456789}\n\n\n\n\nRun with kafka producer\n\n\nBoot kafka as described here:  \nhttp://kafka.apache.org/07/quickstart.html\n, then:\n\n\nbin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \\\n   --producer=kafka --kafka.bootstrap.servers=localhost:9092\n\n\n\n\nThis will start writing to the topic \"maxwell\".", 
+            "text": "Row based replication\n\n\nMaxwell can only operate if row-based replication is on.\n\n\n$ vi my.cnf\n\n[mysqld]\nserver-id=1\nlog-bin=master\nbinlog_format=row\n\n\n\n\nGrant permissions\n\n\nmysql\n GRANT ALL on maxwell.* to 'maxwell'@'%' identified by 'XXXXXX';\nmysql\n GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'%';\n\n# or for running maxwell locally:\n\nmysql\n GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'localhost' identified by 'XXXXXX';\nmysql\n GRANT ALL on maxwell.* to 'maxwell'@'localhost';\n\n\n\n\n\nInstall maxwell\n\n\nYou'll need a version 7 of a JVM.\n\n\ncurl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.2-RC1/maxwell-0.16.2-RC1.tar.gz \\\n       | tar zxvf -\ncd maxwell-0.16.2-RC1\n\n\n\n\nRun with stdout producer\n\n\nUseful for smoke-testing the thing.\n\n\nbin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' --producer=stdout\n\n\n\n\nIf all goes well you'll see maxwell replaying your inserts:\n\n\nmysql\n insert into test.maxwell set id = 5, daemon = 'firebus!  firebus!';\nQuery OK, 1 row affected (0.04 sec)\n\n(maxwell)\n{\ntable\n:\nmaxwell\n,\ntype\n:\ninsert\n,\ndata\n:{\nid\n:5,\ndaemon\n:\nfirebus!  firebus!\n},\nts\n: 123456789}\n\n\n\n\nRun with kafka producer\n\n\nBoot kafka as described here:  \nhttp://kafka.apache.org/07/quickstart.html\n, then:\n\n\nbin/maxwell --user='maxwell' --password='XXXXXX' --host='127.0.0.1' \\\n   --producer=kafka --kafka.bootstrap.servers=localhost:9092\n\n\n\n\nThis will start writing to the topic \"maxwell\".", 
             "title": "Quick Start"
         }, 
         {
@@ -27,7 +27,7 @@
         }, 
         {
             "location": "/compat/", 
-            "text": "Maxwell Compatibility\n\n\n\nRequirements:\n\n\n\n\nJRE 7 or above\n\n\nmysql 5.1, 5.5, 5.6\n\n\nkafka 0.8.2 or greater\n\n\n\n\nUnsupported\n\n\n\n\nMysql 5.7 is untested with Maxwell, and in particular GTID replication is unsupported as of yet.\n\n\nbinlog_row_image=MINIMAL\n is not supported and will break Maxwell in a variety of amusing ways.\n\n\n\n\nMaster recovery\n\n\nCurrently Maxwell is not very smart about master recovery or detecting a promoted slave; if it determines\nthat the server_id has changed between runs, Maxwell will simply delete its old schema cache and binlog position\nand start again.  We plan on improving master recovery in future releases.\n\n\nIf you know the starting position of your new master, you can start the new Maxwell process with the\n\n--init_position\n flag, which will ensure that no gap appears in a master failover.", 
+            "text": "Maxwell Compatibility\n\n\n\nRequirements:\n\n\n\n\nJRE 7 or above\n\n\nmysql 5.1, 5.5, 5.6\n\n\nkafka 0.8.2 or greater\n\n\n\n\nUnsupported\n\n\n\n\nMysql 5.7 is untested with Maxwell, and in particular GTID replication is unsupported as of yet.\n\n\n\n\nbinlog_row_image=MINIMAL\n\n\nAs of 0.17.0, Maxwell supports binlog_row_image=MINIMAL, but it may not be what you want.  It will differ\nfrom normal Maxwell operation in that:\n\n\n\n\nINSERT statements will no longer output a column's default value\n\n\nUPDATE statements will be incomplete; Maxwell outputs as much of the row as given in the binlogs,\n  but \ndata\n will only include what is needed to perform the update (generally, id columns and changed columns).\n  The \nold\n section may or may not be included, depending on the nature of the update.\n\n\nDELETE statements will be incomplete; generally they will only include the primary key.\n\n\n\n\nMaster recovery\n\n\nCurrently Maxwell is not very smart about master recovery or detecting a promoted slave; if it determines\nthat the server_id has changed between runs, Maxwell will simply delete its old schema cache and binlog position\nand start again.  We plan on improving master recovery in future releases.\n\n\nIf you know the starting position of your new master, you can start the new Maxwell process with the\n\n--init_position\n flag, which will ensure that no gap appears in a master failover.", 
             "title": "Compat / Caveats"
         }
     ]

diff --git a/docs/site/quickstart/index.html b/docs/site/quickstart/index.html
@@ -121,9 +121,9 @@ <h3 id="grant-permissions">Grant permissions</h3>
 
 <h3 id="install-maxwell">Install maxwell</h3>
 <p>You'll need a version 7 of a JVM.</p>
-<pre><code>curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.1-RC1/maxwell-0.16.1-RC1.tar.gz \
+<pre><code>curl -sLo - https://github.com/zendesk/maxwell/releases/download/v0.16.2-RC1/maxwell-0.16.2-RC1.tar.gz \
        | tar zxvf -
-cd maxwell-0.16.1-RC1
+cd maxwell-0.16.2-RC1
 </code></pre>
 
 <h3 id="run-with-stdout-producer">Run with stdout producer</h3>

diff --git a/docs/site/sitemap.xml b/docs/site/sitemap.xml
@@ -4,47 +4,47 @@
 
     <url>
      <loc>None/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 
 
 
     <url>
      <loc>None/quickstart/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 
 
 
     <url>
      <loc>None/config/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 
 
 
     <url>
      <loc>None/kafka/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 
 
 
     <url>
      <loc>None/dataformat/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 
 
 
     <url>
      <loc>None/compat/</loc>
-     <lastmod>2015-12-11</lastmod>
+     <lastmod>2015-12-16</lastmod>
      <changefreq>daily</changefreq>
     </url>
 

diff --git a/pom.xml b/pom.xml
@@ -4,7 +4,7 @@
 
   <groupId>com.zendesk</groupId>
   <artifactId>maxwell</artifactId>
-  <version>0.16.1</version>
+  <version>0.16.2</version>
   <packaging>jar</packaging>
 
   <name>maxwell</name>