Added support for reading and writing avro files (#490)

* add support for reading and writing avro files

* Apply suggestions from code review of avro support in pull #490

Set version to 1.4.0 and issue to 490 as suggested in pull request #490

Co-authored-by: Alistair Miles <[email protected]>

Co-authored-by: Juarez Rudsatz <[email protected]>
Co-authored-by: Alistair Miles <[email protected]>

docs/changes.rst

@@ -1,6 +1,16 @@
 Changes
 =======
 
+Version 1.4.0
+-------------
+
+* Added functions :func:`petl.io.avro.fromavro`, :func:`petl.io.avro.toavro`,
+  and :func:`petl.io.avro.appendavro` for reading and writing to 
+  `Apache Avro <https://avro.apache.org/docs/current/spec.html>` files. Avro
+  generally is faster and safer than text formats like Json, XML or CSV.
+  By :user:`juarezr`, :issue:`490`.
+
+
 Version 1.3.0
 -------------
 

docs/io.rst

@@ -288,6 +288,52 @@ Text indexes (Whoosh)
 .. module:: petl.io.sources
 .. _io_helpers:
 
+Avro files (fastavro)
+----------------------------
+
+.. note::
+
+    The following functions require `fastavro
+    <https://github.com/fastavro/fastavro>`_ to be
+    installed, e.g.::
+
+        $ pip install fastavro
+
+.. autofunction:: petl.io.avro.fromavro
+.. autofunction:: petl.io.avro.toavro
+.. autofunction:: petl.io.avro.appendavro
+
+.. literalinclude:: ../petl/test/io/test_avro_schemas.py
+   :name: logical_schema
+   :language: python
+   :caption: Avro schema for logical types 
+   :start-after: begin_logical_schema
+   :end-before: end_logical_schema
+
+.. literalinclude:: ../petl/test/io/test_avro_schemas.py
+   :name: nullable_schema
+   :language: python
+   :caption: Avro schema with nullable fields
+   :start-after: begin_nullable_schema
+   :end-before: end_nullable_schema
+
+.. literalinclude:: ../petl/test/io/test_avro_schemas.py
+   :name: array_schema
+   :language: python
+   :caption: Avro schema with array values in fields
+   :start-after: begin_array_schema
+   :end-before: end_array_schema
+
+.. literalinclude:: ../petl/test/io/test_avro_schemas.py
+   :name: complex_schema
+   :language: python
+   :caption: Example of recursive complex Avro schema
+   :start-after: begin_complex_schema
+   :end-before: end_complex_schema
+
+.. module:: petl.io.avro
+.. _io_avro:
+
 I/O helper classes
 ------------------
 

optional_requirements.txt

@@ -13,3 +13,4 @@ SQLAlchemy==1.3.6
 Whoosh==2.7.4
 xlrd==1.2.0
 xlwt==1.3.0
+fastavro>=0.23.4

petl/io/__init__.py

@@ -35,3 +35,5 @@
     searchtextindexpage, totextindex, appendtextindex
 
 from petl.io.bcolz import frombcolz, tobcolz, appendbcolz
+
+from petl.io.avro import fromavro, toavro, appendavro