Include mitxonline email opt in data

[quazi-h]

Pulling email opt in data from the correct dbt model instead of the data from IRx.

[rachellougee]

See comment below. Also a question, did we not include every model in _irx_mitxonline__models.yml? I don't see irx__mitxonline__openedx__bigquery__email_opt_in in _irx_mitxonline__models.yml for example

[rachellougee]

None of these fields listed here are in int__mitxonline__bulk_email_optin. If you run this model, there should be errors. I think we need to replace all the fields with the correct names.

select
    is_opted_in_for_email
    , course_id
    , full_name
    , preference_set_datetime
    , user_id
from email_opt_in

[quazi-h]

Thanks for catching that, I have updated the column names to the correct ones and also added the yaml for that table. Many of the new models that are not included in the iml file could not be processed by dbt-coves. I was going to write the definitions out manually but since we are tabling this work for now, I figured we could revisit that in the future when this becomes a priority again.

[rachellougee]

LGTM now

[rachellougee]

@quazi-h The new changes look fine. But just wanted to point out that irx__xpro__openedx__bigquery__email_opt_in doesn't really contain the open edx data from xPro. The underlying table raw__irx__edxorg__bigquery__email_opt_in is mostly for edX.org. There are only a couple of xPro courses that are old data in 2018. You probably need to source it directly from raw__xpro__openedx__mysql__bulk_email_optout and then do similar logic as int__mitxonline__bulk_email_optin

[rachellougee]

I asked @pdpinch about xPro email opt in. It sounded like xPro handle this data differently, but no one has asked about it for xPro. So I am not sure how you want to handle irx__xpro__openedx__bigquery__email_opt_in but we probably shouldn't pass around the data he gave to us. That said, it doesn't have to be handled in this PR.

[pdpinch]

What's the status of this? It's been open for a long time.