Skip to content

Commit

Permalink
Recover Morningstar generic mode with morningstar-screener
Browse files Browse the repository at this point in the history
  • Loading branch information
@joseballester
joseballester committed Oct 5, 2022
1 parent 9617251 commit f941334
Show file tree
Hide file tree
Showing 6 changed files with 391 additions and 18 deletions.
1 change: 1 addition & 0 deletions docs/behavior.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ <h2 class="mb-5">How µFunds works</h2>
<p>In a few words, µFunds works by fetching an HTML page from the chosen data source, parsing the asked data in it and just returning it back to you. More concretely, the performed steps are the following:</p>
<ol>
<li>The user calls the muFunds function from a Google Sheets spreadsheet for a certain <code>option</code> (asset attribute), <code>id</code> (asset identifier) and <code>source</code> (website where fetch data from).</li>
<li>If <code>source</code> is explicitly given by the user, then µFunds plans to fetch a the asset detail page for that <code>id</code>. Otherwise, µFunds plans to fetch a Morningstar page in generic mode (<code>morningstar</code>).</li>
<li>If the page is cached by Google Apps Script's cache from the last 2 hours, then no external call is performed. Otherwise, the page is fetched and saved in Google Apps Script's cache for the next 2 hours.</li>
<li>The asked attribute is parsed from the Morningstar page's HTML and returned.</li>
</ol>
Expand Down
168 changes: 168 additions & 0 deletions docs/compatibility.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -452,48 +452,111 @@ <h2 class="mb-5">Compatibility list</h2>
<td>Morningstar (Australia)</td>
<td><code>morningstar-au</code></td>
<td>APIR</td>
</tr>
<tr>
<td>Austria</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Austria</td>
<td>Mutual funds</td>
<td>Morningstar (Austria)</td>
<td><code>morningstar-at</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Belgium</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Belgium</td>
<td>Mutual funds</td>
<td>Morningstar (Belgium)</td>
<td><code>morningstar-be</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Denmark</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Denmark</td>
<td>Mutual funds</td>
<td>Morningstar (Denmark)</td>
<td><code>morningstar-dk</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Finland</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Finland</td>
<td>Mutual funds</td>
<td>Morningstar (Finland)</td>
<td><code>morningstar-fi</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>France</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>France</td>
<td>Mutual funds</td>
<td>Morningstar (France)</td>
<td><code>morningstar-fr</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Germany</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Germany</td>
<td>Mutual funds</td>
<td>Morningstar (Germany)</td>
<td><code>morningstar-de</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Hong Kong</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Hong Kong</td>
<td>ETFs</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID<br>SEHK Stock Code</td>
</tr>
<tr>
<td>Iceland</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Iceland</td>
Expand All @@ -508,27 +571,55 @@ <h2 class="mb-5">Compatibility list</h2>
<td colspan="2">Google Finance</td>
<td class="hidden"></td>
<td><code>MUTF_IN:</code> + ticker</td>
</tr>
<tr>
<td>Ireland</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Ireland</td>
<td>Mutual funds</td>
<td>Morningstar (Ireland)</td>
<td><code>morningstar-ie</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Italy</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Italy</td>
<td>Mutual funds</td>
<td>Morningstar (Italy)</td>
<td><code>morningstar-it</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Norway</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Norway</td>
<td>Mutual funds</td>
<td>Morningstar (Norway)</td>
<td><code>morningstar-no</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Netherlands</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Netherlands</td>
Expand All @@ -543,27 +634,83 @@ <h2 class="mb-5">Compatibility list</h2>
<td>Morningstar (Australia)</td>
<td><code>morningstar-au</code></td>
<td>APIR</td>
</tr>
<tr>
<td>Portugal</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Portugal</td>
<td>Mutual funds</td>
<td>Morningstar (Portugal)</td>
<td><code>morningstar-pt</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Singapore</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Singapore</td>
<td>ETFs</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID<br>SGX Stock Code</td>
</tr>
<tr>
<td>Singapore</td>
<td>Insurance Linked Funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>Morningstar ID</td>
</tr>
<tr>
<td>South Africa</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>South Africa</td>
<td>Mutual funds</td>
<td>Morningstar (South Africa)</td>
<td><code>morningstar-za</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Spain</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Spain</td>
<td>Mutual funds</td>
<td>Morningstar (Spain)</td>
<td><code>morningstar-es</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Spain</td>
<td>ETFs</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID<br>BME Ticker</td>
</tr>
<tr>
<td>Spain</td>
<td>Pension plans</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>Morningstar ID</td>
</tr>
<tr>
<td>Spain</td>
Expand All @@ -578,20 +725,41 @@ <h2 class="mb-5">Compatibility list</h2>
<td>Quefondos.com</td>
<td><code>quefondos</code></td>
<td>DGS Code</td>
</tr>
<tr>
<td>Switzerland</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>Switzerland</td>
<td>Mutual funds</td>
<td>Morningstar (Switzerland)</td>
<td><code>morningstar-ch</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>United Kingdom</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>United Kingdom</td>
<td>Mutual funds</td>
<td>Morningstar (United Kingdom)</td>
<td><code>morningstar-gb</code></td>
<td>ISIN<br>Morningstar ID</td>
</tr>
<tr>
<td>United States</td>
<td>Mutual funds</td>
<td>Morningstar (generic)</td>
<td><code>morningstar</code></td>
<td>ISIN<br>Ticker<br>CUSIP</td>
</tr>
<tr>
<td>United States</td>
Expand Down
7 changes: 2 additions & 5 deletions docs/troubleshooting.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,22 +69,19 @@ <h2 class="mb-5">Troubleshooting</h2>
<p><b>Error: Asset identifier is empty.</b></p>
<p>Please check that you introduced an asset identifier (string) as the second argument of the <code>muFunds</code> function.</p>
</li>
<li>
<p><b>Data source is required as third argument. Please see www.mufunds.com/usage.html</b></p>
<p>Please check that you introduced a valid data source as the third argument of the <code>muFunds</code> function. Check the <a href="usage.html">Usage and examples</a> documentation page to learn about the available options.</p>
<p>Note: The data source argument is required from June 24, 2022, since country-agnostic Morningstar reports from <code>quotes.morningstar.com</code> (used when no source was explicitly defined) stopped working on June 23, 2022.</p>
</li>
<li>
<p><b>Error: Source is not compatible. Please check the documentation for the compatibility list.</b></p>
<p>Please check the <a href="usage.html">Usage and examples</a> and <a href="compatibility.html">Compatibility list</a> documentation pages to verify that the data source is compatible with µFunds. Remember that, if you want to obtain data from Google Finance, you should use the <a href="https://support.google.com/docs/answer/3093281">Google Finance function included in Google Sheets</a> instead.</p>
</li>
<li>
<p><b>Error: Wrong combination of asset identifier and source. Please check the accepted ones at the documentation.</b></p>
<p>Please check the <a href="compatibility.html">Compatibility list</a> documentation pages to verify that the asset is compatible with the data source you have chosen. This error is due to an exception when fetching the data page, for example a <i>Not Found</i> (404) code.</p>
<p>If you were using a generic Morningstar (<code>morningstar</code>) call, consider using a country-specific page (<code>morningstar-**</code>).</p>
</li>
<li>
<p><b>Error: (A certain option) is not available for this asset and source. Please try another data source.</b></p>
<p>Some data sources do not offer all the data options that are possible to call via µFunds. In this case, consider using an alternative data source for your same asset, as sometimes more than one source is compatible at the same time.</p>
<p>If you were using a generic Morningstar (<code>morningstar</code>) call, consider using a country-specific page (<code>morningstar-**</code>).</p>
</li>
<li>
<p><b>Error: Morningstar ID search failed. Try using the asset's Morningstar ID or another compatible data source.</b></p>
Expand Down
19 changes: 8 additions & 11 deletions docs/usage.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,18 +60,9 @@ <h5 class="mb-3"><a href="index.html#guides">List of guides</a></h5>
<h2 class="mb-5">Usage and examples</h2>
</div>
<div class="col-12 mb-1 mb-md-0">
<div class="alert alert-warning mb-5">
<p>
<b>Important:</b> Country-agnostic Morningstar reports from <code>quotes.morningstar.com</code> stopped working on June 23, 2022. Therefore, from June 24, 2022, µFunds requires a data source as the third argument of the function call.
</p>

<p class="mb-0">
You should now call µFunds with <code>=muFunds(option, id, "morningstar-**")</code>, where <code>**</code> represents the 2-letter code of a country where your asset is available for sale. In most cases, you can just use the country where you live. Please check the list of available countries and codes in the <a href="#sources">Data sources</a> section below.
</p>
</div>
<h5 class="mb-4">Function call</h5>
<p>µFunds makes you able to use the <code>muFunds</code> function in any cell within your spreadsheet. This function will return the asked value based on three arguments, so you can call it as</p>
<p class="lead text-center"><code class="formula rounded bg-light p-2">=muFunds(option, id, source)</code> <small>(if you use a <i>period</i> as decimal separator)</small></p>
<p>µFunds makes you able to use the <code>muFunds</code> function in any cell within your spreadsheet. This function will return the asked value based on three arguments, so you can call it as</p>
<p class="lead text-center"><code class="formula rounded bg-light p-2">=muFunds(option, id, source)</code> <small>(if you use a <i>period</i> as decimal separator)</small></p>
<p class="lead text-center"><code class="formula rounded bg-light p-2">=muFunds(option; id; source)</code> <small>(if you use a <i>comma</i> as decimal separator)</small></p>
<p>where <code>option</code> is a valid <a href="#options">returned data option</a>, <code>id</code> is an <a href="#ids">asset identifier</a> (e.g. ISIN, ticker or Morningstar ID) and <code>source</code> is an identifier for the <a href="#sources">data source</a> where you want to get the data from.</p>
<p>This function call will return the latest information available from the chosen source. Most of these sources don't offer historical values, which makes comparisons and backtesting difficult. If interested, you can consider <a href="history.html">saving a history of daily data to your own spreadsheet using a Google Apps Script function and a time-driven trigger</a> (link to guide).</p>
Expand Down Expand Up @@ -129,7 +120,13 @@ <h5 class="mb-4">Asset identifiers (<code>id</code>)</h5>
<hr class="mt-4 mb-5"/>
<h5 class="mb-4">Data sources (<code>source</code>)</h5>
<p>Although µFunds was originally designed to import data from Morningstar, now it is compatible with other information sources that potentially offer more updated or reliable data in certain cases. This is performed, as of now, via HTML parsing.</p>
<p>If no source is defined, then the asked information will be loaded from Morningstar in generic mode.</p>
<ul>
<li>
<p><b>Morningstar generic mode</b> (<code>morningstar</code>)</p>
<p>This source relies on Morningstar screener pages, that happen to be the easiest way to get information from an asset, regardless of the used identifier, as they work with ISINs, mutual fund tickers, ETF tickers, Morningstar IDs, etc.</p>
<p>Morningstar screener info is not a user-friendly interface and therefore loading times are faster. Furthermore, no country needs to be chosen and no translation from standard identifiers to Morningstar IDs is required. On the other hand, expenses data is usually not available and sometimes data is provided in a different currency of the asset's, is provided incorrectly or is not provided at all. In these cases, Morningstar country-specific mode should be used.</p>
</li>
<li>
<p><b>Morningstar country pages</b> (<code>morningstar-**</code>)</p>
<p>µFunds visits Morningstar snapshot pages and simulates the human behavior when checking for the data manually in a Morningstar website for a certain country. All attributes are up-to-date and reliable, including expense ratios, as the information is the one provided directly to Morningstar's visitors.</p>
Expand Down
4 changes: 2 additions & 2 deletions main.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ function muFunds(option, id, source) {
throw new Error( "Asset identifier is empty." );
}

if (!source) {
throw new Error( "Data source is required as third argument. Please see www.mufunds.com/usage.html" );
if (!source || source === 'morningstar') {
return loadFromMorningstarScreener(option, id);
}

// Input already validated
Expand Down
Loading

0 comments on commit f941334

Please sign in to comment.