Disruption of Quandl data, esp. CME futures

[rsvp]

Data from Quandl failing -- NASDAQ takeover

For example, when retrieving the Dec 2019 Eurodollar futures contact:

>>> tmp = fe.get('CME/EDZ2019')
ValueError: INVALID symbol string or code variable.

Projects fecon235 and fecon236 have many data dependencies on the Quandl API, including but limited to, futures quotes.

• Bug
• Enhancement
• Upstream disruption

Steps which reproduce behavior

1. Eurodollar futures quotes relies on CME vendor at Quandl.
2. "PAGE NO LONGER EXISTS." at usual vendor page: https://www.quandl.com/data/CME-Chicago-Mercantile-Exchange-Futures-Data/documentation/data-organization
3. Last successful Google crawl on said vendor page was December 1, 2018.
4. Top notification, "Quandl has been acquired by Nasdaq. Learn more" links to a Press Release from Nasdaq: https://business.nasdaq.com/mediacenter/pressreleases/1855930/nasdaq-acquires-quandl-to-advance-the-use-of-alternative-data which includes the following statements:

December 4, 2018, Nasdaq Acquires Quandl

Nasdaq (Nasdaq: NDAQ) announced today it has acquired Quandl, Inc., a leading provider of alternative and core financial data. Quandl from over 350 sources to more than 30,000 active monthly users. The company offers a global database of alternative, financial and public data, including information on capital markets, energy, shipping, healthcare, education, demography, economics and society.

Nasdaq plans to combine Quandl with its existing Analytics Hub business within Global Information Services.

Terms of the deal were not disclosed, but this investment is consistent with both Nasdaq’s strategy to maximize opportunities as a technology and analytics provider to capital markets, as well as its capital deployment and return on investment capital objectives.

Matthew Sheahan
Phone Number: + 1 212 231 5945
Email: [email protected]

5. Search for "CME" on Quandl's search page https://www.quandl.com/search no longer returns any results for futures contracts (bonds, FX, metals, etc.) directly provided by CME.
6. Google search leads to link for "CME Dec 2019 Eurodollar Futures", https://www.quandl.com/CME/EDZ2019 -- but currently displays "PAGE NO LONGER EXISTS." Last successful Google crawl was on November 30, 2018.
7. Disappearance of CME data thus most likely occurred between November 30 and December 7, 2018. Midway, on December 4, Nasdaq announced acquisition of Quandl, a distributor of CME data.

Additional helpful details for bugs

• Problem started recently, but not in older versions

• Problem happens with all files, not only some files

• Problem can be reliably reproduced

• Problem happens randomly

• fecon236 version: 10.8.0

• Python version: 2.7 and 3 series

• Operating system: All

Until advised otherwise, we will assume that the Quandl data will be behind paywalls (Nasdaq's Analytics Hub).

[rsvp]

Regarding "Nasdaq Analytics Hub"

May 16, 2017 (GLOBE NEWSWIRE) -- Nasdaq (Nasdaq:NDAQ), a leading provider of real-time market data and analytics to financial professionals and retail investors globally, today announced the launch of the "Nasdaq Analytics Hub," a new platform that provides the buy side with investment signals that are derived from structured and unstructured data, and unique to Nasdaq. Today, the platform analyzes social media sentiment, central bank communications, retail sentiment, technical factors, and event based signals.

Advances in technology and automation are increasing the demand for alternative data sets to help asset and hedge fund managers, algorithmic traders and active managers make informed investment decisions. The Nasdaq Analytics Hub is the second product to launch from the Innovation Lab to address this evolution, continuing Nasdaq's expansion into real-time and daily analytics that help market participants augment their strategies.

Innovation Lab's motto: "Building new capabilities through machine intelligence and proprietary at-source and third-party data for the future." https://business.nasdaq.com/intel/GISLab/GISLab.html

The Nasdaq Analytics Hub provides a growing library of alternative and financial data that can be used to improve investment strategies. The data sets are normalized, validated and back-tested by our artificial intelligence engine to ensure the resulting outputs translate into real-life use cases that can easily be incorporated into the investment management process. It allows firms to integrate multiple data sets into portfolio allocation decisions and research without having to set up vendor relationships and contracts with different data providers. Non-traditional, unstructured data can be leveraged to gain insights on financial markets and allow investors to better manage risk and find alpha.

Nasdaq Analytics Hub: https://business.nasdaq.com/intel/GIS/Analytics-Hub.html

That Nasdaq service appears to be proprietary, and is not transparent about costs. There is no mention of a publicly available API, only a contact form to "request a demo." The fate of datasets uploaded to Quandl is dark. An apology or explanation for disrupted Quandl users has not been offered by Nasdaq thus far.

[rsvp]

From: Quandl CEO, Dec 04 2018

Our mission has always been to enable data-driven investing, and now, with Nasdaq’s institutional backing, we will be able to grow and scale faster, with more data and richer functionality on Quandl. Nasdaq’s unique position within the global capital markets will bring us reach, assets, and technology so we can better serve you.

Everything is business as usual: our data, our APIs, our website, our services — everything will remain as is and will keep getting better in the future. We will continue to operate under the Quandl brand, with Quandl technology, people, and processes. I will continue to run the business with my co-founder, Abraham Thomas, and your points of contact will remain the same. As before and doubly so now, everything we do will be driven by our passion to continually improve our platform and grow our suite of datasets.

Sincerely,
Tammer Kamel
CEO | Quandl

Source: https://blog.quandl.com/quandl-acquired-by-nasdaq-to-advance-use-of-alternative-data

The date of Kamel's blog post is the same as the Nasdaq press release, so we are puzzled: is the Quandl API still functional?

>>> xbt = fe.get('BCHAIN/MKPRU')
>>> fe.tail(xbt)
                      Y
T                      
2018-12-01  4106.871667
2018-12-02  4116.777500
2018-12-03  4167.546667
2018-12-04  3967.524167
2018-12-05  3961.493333
2018-12-06  3858.349167
2018-12-07  3742.943333

OK, so our Quandl module works for Bitcoin quotes on 2018-12-08 20:56:15 UTC.
But why not for quotes on futures from the CME, Chicago Mercantile Exchange?
Did they pull out of Quandl due to Nasdaq takeover, or were they suppressed?

CME Group is an important source of data since it also includes the following exchanges: CBOT, NYMEX and COMEX. Contact info: +1 312 930 1000

---

Addendum: U.S. CFTC Reports on Weekly Commitment of Futures Traders appears intact and free at Quandl, https://www.quandl.com/data/CFTC-Commodity-Futures-Trading-Commission-Reports -- our notebook https://git.io/cotr discerns how various asset classes are positioned in the market.

[rsvp]

Alternate services direct from CME

It's evident that CME is also interested in monetizing their data in the way of licensing and data fees: https://www.cmegroup.com/market-data.html

Real-time Market Data

• Current bid/ask and price quotes
• Opening & closing ranges
• High-Low Prices
• Last Sale Prices, Settlement prices
• Depth of Book
• Open Outcry
• RFQs
• Price Limits
• See https://www.cmegroup.com/market-data/real-time-market-data.html

"Datamine" Historical Data > 450 terabytes

• Market By Order, Market Depth, Top-of-Book (BBO)
• Time & Sales, End-of-Day
• Block Trades
• See https://www.cmegroup.com/market-data/datamine-historical-data.html

Liquidity tool (as of September 2018)

The data science and analytics technology underpinning the CME Liquidity Tool processes more than 70 billion records per month. Liquidity Tool offers a way to view and export charts measuring market liquidity on a daily, weekly, monthly, quarterly and yearly basis, during periods of varying market volatility across North American, European and Asian trading hours. The tool measures market liquidity based on three key measures of market liquidity and cost:

• Bid-ask spread, or the difference between the prices quoted to buy and sell futures;
• Book depth, which is the number of contracts in both buy and sell orders at a given price; and
• Cost to trade, which is the amount it will cost in ticks equal to the minimum bid-ask spread to buy or sell a specific order size.
• See https://www.cmegroup.com/tools-information/cme-liquidity-tool.html

The tool is available to any user who signs up for a free CMEGroup.com account. CME Group exchanges offer the widest range of global benchmark products across all major asset classes, including futures and options based on interest rates, equity indexes, foreign exchange, energy, agricultural products and metals. Around the world, CME Group brings buyers and sellers together through its CME Globex® electronic trading platform. CME Group also operates one of the world's leading central counterparty clearing providers through CME Clearing, which offers clearing and settlement services across asset classes for exchange-traded and over-the-counter derivatives.

Recent remarks

As @econdb mentioned, "CME and NASDAQ are competitors, so the conditions by which the CME was licensed to Quandl might not apply anymore."