Generate null values for nullable fields

[nightscape]

This PR changes the behavior of DataframeGenerator to generate null values if a field is nullable.
This solves #302.
There is one rather ugly spot in the code:

colValues.contains(null) ||
  // Unfortunately, dataframeGen.arbitrary sometimes generates DataFrames where all
  // rows have exactly identical values.
  // In that case, even generating many rows doesn't help to get some nulls...
  // To work around this we check if we generated at least some distinct values.
  colValues.distinct.size < 4 ||
  // This is needed for Array-valued fields where .distinct returns all values, even when
  // they're identical.
  colValues.size == colValues.distinct.size

If someone knows how to prevent the generator from generating absolutely identical rows this would simplify the code a lot.

[holdenk]

If you've got the time to look at the felling test let me know, otherwise I can take a look.

[codecov-io]

Codecov Report

Merging #303 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #303      +/-   ##
==========================================
+ Coverage   86.36%   86.39%   +0.02%     
==========================================
  Files          46       46              
  Lines        1005     1007       +2     
  Branches       86       87       +1     
==========================================
+ Hits          868      870       +2     
  Misses        116      116              
  Partials       21       21

Flag	Coverage Δ
#python	85.87% <ø> (ø)	⬆️
#scala	71.01% <100%> (+0.07%)	⬆️

Impacted Files	Coverage Δ
...holdenkarau/spark/testing/DataframeGenerator.scala	97.56% <100%> (+0.12%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f130b0...7abeb87. Read the comment docs.

[nightscape]

@holdenk fixed the failing test. I needed to explicitly set some fields to nullable = false in another test.

[nightscape]

@holdenk ping 😉

[holdenk]

Sorry for the delay, and thanks for adding this!

[holdenk]

I'll try and do a release next week so folks can start using it :) Thank you @nightscape and thanks for working with my slow review cycle.

[nightscape]

Thanks for merging! And no worries about review cycles, I'm no better 😉