Make stack create CategoricalVector{String} vector from column names

[bkamins]

Currently it is Vector{Symbol}. See JuliaInterop/RCall.jl#331 for a discussion (but I think this would be also useful in general).

[bkamins]

@nalimilan - given #2029 what do you think we should do here?
We can use PooledVector instead of CategoricalVector if we want to cut down the dependency on CategoricalArrays.jl. I planned to implement this change this week.

[nalimilan]

Ah, so indeed that would be a reason to keep depending on CategoricalArrays. CategoricalVector{String} would be better than PooledVector{String} here to preserve the ordering of columns.

For reference, in the tidyverse, tidyr::gather uses factor_key = FALSE by default, while its predecessors reshape2::melt and utils::stack did the reverse. The discussion about the change at tidyverse/tidyr#96. I must say it's a bit surprising as the only justification seems to be that joining printed a silly warning about mixing factors and characters. I guess a more general reason could be to keep things simple by default. OTOH I could find several examples using factor_key=TRUE on the Web, so it looks like it's used.

[bkamins]

In R I think that the problem is that "factor" is a bit broken. We do not have this problem in CategoricalArrays.jl.

Now as:

julia> @time using DataFrames
  1.334656 seconds (1.77 M allocations: 106.438 MiB, 0.28% gc time)

I am OK with this loading time, so as noted earlier I have no problem with keeping CategoricalArrays.jl (removing it would cut it to probably something around 0.7 seconds at best, so the perception of the lag would be of similar order I think).

@nalimilan - so it is a call to you to make a decision here 😄.

@quinnj - maybe you want to comment here or in #2029 how you see the CategoricalArrays.jl case (keep it or drop it?), especially in relation to CSV.jl. Thank you!

[nalimilan]

I think people complain not only about the loading time, but also about the fact that loading CategoricalArrays invalidates lots of Base functions, which introduces many delays when calling the function for the first time in the session. But yeah, it sounds annoying to have to drop CategoricalArrays just because of that.

[quinnj]

Yeah, I like the functionality of CategoricalArrays.jl, but I wish it didn't cause so much compiler trouble; it'd be great if we could figure all that out quickly to make it an easier/lighter/more friendly dependency.

[nalimilan]

Unfortunately I don't know how...

[bkamins]

@quinnj - do you have some time to spare and lay out what are the core problems that CategoricalArrays.jl causes for compiler (i.e. I know the design of the package but I never studied what exactly causes the most pain). Thank you!

[quinnj]

From what I understand, it comes down to some problematic convert definitions, like convert(::Type{Any}, x::CategoricalValue), which, when defined, invalidate basically any other convert method, triggering them all to need to recompile. That's why I opened JuliaLang/julia#32613 as a way to allow a wrapper type like CategoricalValue to not need such problematic convert methods.