Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform dayofyear-based calculations according to UTC, not local time #2055

Merged
merged 55 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
6c4c32a
Add clearsky.ipynb (Remove before merge)
yhkee0404 May 19, 2024
9ce02d5
Fix clearsky.lookup_linke_turbidity to be timezone-aware
yhkee0404 May 19, 2024
0ad7a68
Fix irradiance.get_extra_radiation to be timezone-aware
yhkee0404 May 19, 2024
05961d2
Rerun and clear errors in clearsky.ipynb
yhkee0404 May 19, 2024
45a36cb
Remove clearsky.ipynb
yhkee0404 May 19, 2024
b00a2d4
Update what's new
yhkee0404 May 19, 2024
7bb7e02
Merge branch 'main' into clearsky-timezone
yhkee0404 May 23, 2024
6d1ab57
Merge branch 'main' into clearsky-timezone
yhkee0404 May 26, 2024
3fed282
Add docstring Notes to lookup_linke_turbidity
yhkee0404 Jun 1, 2024
6f98df7
Add tools._pandas_to_utc
yhkee0404 Jun 1, 2024
fe5b5c1
Update what's new
yhkee0404 Jun 4, 2024
97afcb8
Add docstring Notes to _pandas_to_doy
yhkee0404 Jun 5, 2024
defddfd
Revert _datetimelike_scalar_to_datetimeindex
yhkee0404 Jun 5, 2024
a08ca79
Revert tests.test_irradiance.times
yhkee0404 Jun 5, 2024
6baa1ed
Replace lowercase utc with uppercase
yhkee0404 Jun 5, 2024
4515366
Fix clearsky lookup_linke_turbidity and _interpolate_turbidity
yhkee0404 Jun 5, 2024
83a61ba
Fix linter
yhkee0404 Jun 8, 2024
745579c
Fix clearsky._interpolate_turbidity to keep timezone
yhkee0404 Jun 8, 2024
b6d238a
Revert irradiance.py
yhkee0404 Jun 8, 2024
fa62003
Clarify UTC input of ephem.Date
yhkee0404 Jun 8, 2024
5d80876
Fix tz-naive delta_t of dni_extra nrel
yhkee0404 Jun 8, 2024
3e95694
Fix tz-naive delta_t of get_solarposition nrel_numpy
yhkee0404 Jun 8, 2024
59cb82b
Avoid future midnight of location.get_sun_rise_set_transit spa
yhkee0404 Jun 8, 2024
6e1c251
Fix tz-naive delta_t of location.get_sun_rise_set_transit spa
yhkee0404 Jun 8, 2024
0baa338
Fix local midnight of hour_angle and sun_rise_set_transit
yhkee0404 Jun 8, 2024
cd45c74
Fix dayofyear of solarposition decl and eot, i.e. clearsky.bird
yhkee0404 Jun 8, 2024
1e6914d
Fix dayofyear of spectrum.spectrl2 dni_extra spencer
yhkee0404 Jun 8, 2024
dd78f3e
Fix monthly SeasonalTiltMount orientation
yhkee0404 Jun 8, 2024
35e0756
Fix year and dayofyear of solrad
yhkee0404 Jun 8, 2024
758ea98
Fix flake8
yhkee0404 Jun 15, 2024
5102347
Merge branch 'main' into clearsky-timezone
yhkee0404 Jun 15, 2024
1c940fb
Merge branch 'main' of https://github.com/pvlib/pvlib-python into cle…
cwhanse Jul 1, 2024
12c6a49
resolve whatsnew
cwhanse Jul 1, 2024
ab7d307
don't change v0.11.0
cwhanse Jul 1, 2024
82aa0df
Merge branch 'main' into clearsky-timezone
cwhanse Aug 12, 2024
c4862b5
minor adjustment to test_clearsky
cwhanse Aug 12, 2024
57d052e
undo edit to get_solrad
cwhanse Aug 12, 2024
85eae01
undo edit to get_solrad
cwhanse Aug 12, 2024
3fc5579
Merge branch 'main' of https://github.com/pvlib/pvlib-python into cle…
cwhanse Sep 4, 2024
f255e8b
Update pvlib/clearsky.py
cwhanse Sep 4, 2024
63df590
Update docs/sphinx/source/whatsnew/v0.11.1.rst
cwhanse Sep 4, 2024
639ce7f
Update docs/sphinx/source/whatsnew/v0.11.1.rst
cwhanse Sep 4, 2024
33f778f
Update docs/examples/irradiance-transposition/plot_seasonal_tilt.py
yhkee0404 Sep 6, 2024
16f0e5a
Revert benchmarks/benchmarks/solarposition.py
yhkee0404 Sep 9, 2024
2745954
Revert test_lookup_linke_turbidity_nointerp
yhkee0404 Sep 9, 2024
eb2707f
Revert solrad.py
yhkee0404 Sep 9, 2024
ab953b5
Revert plot_seasonal_tilt.py except notes
yhkee0404 Sep 9, 2024
b8246c0
Update test_sun_rise_set_transit_spa with delta_t
yhkee0404 Sep 12, 2024
7da25f4
Update test_nrel_earthsun_distance with delta_t
yhkee0404 Sep 12, 2024
70d06cb
Update test_sun_rise_set_transit_geometric with naive tz
yhkee0404 Sep 12, 2024
dea79d3
Update test_hour_angle with inversion and naive tz
yhkee0404 Sep 12, 2024
2143c94
Remove _is_leap_year from _interpolate_turbidity
yhkee0404 Sep 12, 2024
80a6afb
Merge branch 'main' into clearsky-timezone
yhkee0404 Sep 12, 2024
87cd719
Merge branch 'main' into clearsky-timezone
yhkee0404 Sep 18, 2024
c4b65e1
Update docs/sphinx/source/whatsnew/v0.11.1.rst
yhkee0404 Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions benchmarks/benchmarks/solarposition.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import datetime
import pandas as pd
import pvlib
from pvlib import solarposition
from pvlib import solarposition, tools

from packaging.version import Version

Expand Down Expand Up @@ -51,7 +51,7 @@ def time_sun_rise_set_transit_ephem(self, ndays):
self.times_daily, self.lat, self.lon)

def time_sun_rise_set_transit_geometric_full_comparison(self, ndays):
dayofyear = self.times_daily.dayofyear
dayofyear = tools._pandas_to_doy(self.times_daily)
yhkee0404 marked this conversation as resolved.
Show resolved Hide resolved
declination = solarposition.declination_spencer71(dayofyear)
equation_of_time = solarposition.equation_of_time_spencer71(dayofyear)
solarposition.sun_rise_set_transit_geometric(
Expand Down
5 changes: 3 additions & 2 deletions docs/examples/irradiance-transposition/plot_seasonal_tilt.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# with :py:class:`~pvlib.modelchain.ModelChain`.

import pvlib
from pvlib import pvsystem, location, modelchain, iotools
from pvlib import pvsystem, location, modelchain, iotools, tools
from pvlib.temperature import TEMPERATURE_MODEL_PARAMETERS
import pandas as pd
import pathlib
Expand All @@ -32,7 +32,8 @@ class SeasonalTiltMount(pvsystem.AbstractMount):
surface_azimuth: float = 180.0

def get_orientation(self, solar_zenith, solar_azimuth):
tilts = [self.monthly_tilts[m-1] for m in solar_zenith.index.month]
tilts = [self.monthly_tilts[m-1]
for m in tools._pandas_to_utc(solar_zenith.index).month]
yhkee0404 marked this conversation as resolved.
Show resolved Hide resolved
return pd.DataFrame({
'surface_tilt': tilts,
'surface_azimuth': self.surface_azimuth,
Expand Down
6 changes: 6 additions & 0 deletions docs/sphinx/source/whatsnew/v0.11.1.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ Enhancements

Bug fixes
~~~~~~~~~
* Changes clearsky.lookup_linke_turbidity and irradiance.get_extra_radiation to use UTC instead of local
timezone when determining day of year. pandas does not account for timezone when computing day of year.
Instead, pandas returns the DOY using only the time part of the datetime. As a consequence the same
datetime with different localizations can yield different DOY values, which can cause small inconsistencies
in some solar position and clear sky calculations. (:issue:`2054`, :pull:`2055`)
yhkee0404 marked this conversation as resolved.
Show resolved Hide resolved

yhkee0404 marked this conversation as resolved.
Show resolved Hide resolved

Testing
Expand Down Expand Up @@ -61,6 +66,7 @@ Requirements
Contributors
~~~~~~~~~~~~
* Echedey Luis (:ghuser:`echedey-ls`)
* Yunho Kee (:ghuser:`yhkee0404`)
* Chris Deline (:ghuser:`cdeline`)
* Ioannis Sifnaios (:ghuser:`IoannisSifnaios`)
* Leonardo Micheli (:ghuser:`lmicheli`)
Expand Down
24 changes: 13 additions & 11 deletions docs/tutorials/irradiance.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
"This tutorial requires pvlib >= 0.6.0.\n",
"\n",
"Authors:\n",
"* Will Holmgren (@wholmgren), University of Arizona. July 2014, April 2015, July 2015, March 2016, July 2016, February 2017, August 2018."
"* Will Holmgren (@wholmgren), University of Arizona. July 2014, April 2015, July 2015, March 2016, July 2016, February 2017, August 2018.\n",
"* Yunho Kee (@yhkee0404), Haezoom. Jun 2024."
]
},
{
Expand Down Expand Up @@ -393,6 +394,7 @@
"source": [
"tus = pvlib.location.Location(32.2, -111, 'US/Arizona', 700, 'Tucson')\n",
"times = pd.date_range(start='2016-01-01', end='2016-01-02', freq='1min', tz=tus.tz)\n",
"times_utc = times.tz_convert('UTC')\n",
"ephem_data = tus.get_solarposition(times)\n",
"irrad_data = tus.get_clearsky(times)\n",
"irrad_data.plot()\n",
Expand Down Expand Up @@ -810,7 +812,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"reindl_diffuse = pvlib.irradiance.reindl(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], irrad_data['ghi'], dni_et,\n",
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
Expand Down Expand Up @@ -858,7 +860,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"reindl_diffuse = pvlib.irradiance.reindl(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], irrad_data['ghi'], dni_et,\n",
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
Expand Down Expand Up @@ -913,7 +915,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"\n",
"haydavies_diffuse = pvlib.irradiance.haydavies(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], dni_et,\n",
Expand Down Expand Up @@ -967,7 +969,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"\n",
"haydavies_diffuse = pvlib.irradiance.haydavies(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], dni_et,\n",
Expand Down Expand Up @@ -1028,7 +1030,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"\n",
"haydavies_diffuse = pvlib.irradiance.haydavies(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], dni_et,\n",
Expand Down Expand Up @@ -1067,7 +1069,7 @@
"sun_az = ephem_data['azimuth']\n",
"DNI = irrad_data['dni']\n",
"DHI = irrad_data['dhi']\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"AM = pvlib.atmosphere.get_relative_airmass(sun_zen)\n",
"\n",
"surf_tilt = 32\n",
Expand Down Expand Up @@ -1316,7 +1318,7 @@
"sun_az = ephem_data['azimuth']\n",
"DNI = irrad_data['dni']\n",
"DHI = irrad_data['dhi']\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"AM = pvlib.atmosphere.get_relative_airmass(sun_zen)\n",
"\n",
"surf_tilt = 32\n",
Expand All @@ -1330,7 +1332,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"\n",
"haydavies_diffuse = pvlib.irradiance.haydavies(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], dni_et,\n",
Expand Down Expand Up @@ -1371,7 +1373,7 @@
"sun_az = ephem_data['azimuth']\n",
"DNI = irrad_data['dni']\n",
"DHI = irrad_data['dhi']\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"DNI_ET = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"AM = pvlib.atmosphere.get_relative_airmass(sun_zen)\n",
"\n",
"surf_tilt = 32\n",
Expand All @@ -1385,7 +1387,7 @@
" ephem_data['apparent_zenith'], ephem_data['azimuth'])\n",
"klucher_diffuse.plot(label='klucher diffuse')\n",
"\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times.dayofyear)\n",
"dni_et = pvlib.irradiance.get_extra_radiation(times_utc.dayofyear)\n",
"\n",
"haydavies_diffuse = pvlib.irradiance.haydavies(surf_tilt, surf_az, \n",
" irrad_data['dhi'], irrad_data['dni'], dni_et,\n",
Expand Down
17 changes: 13 additions & 4 deletions pvlib/clearsky.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,13 @@
Returns
-------
turbidity : Series

Notes
-----
Linke turbidity is obtained from a file of historical monthly averages.
The returned value for each time is either the monthly value or an
interpolated value to smooth the transition between months.
Interpolation is done on the day of year as determined by UTC.
"""

# The .h5 file 'LinkeTurbidities.h5' contains a single 2160 x 4320 x 12
Expand Down Expand Up @@ -201,7 +208,7 @@
if interp_turbidity:
linke_turbidity = _interpolate_turbidity(lts, time)
else:
months = time.month - 1
months = tools._pandas_to_utc(time).month - 1
linke_turbidity = pd.Series(lts[months], index=time)

linke_turbidity /= 20.
Expand Down Expand Up @@ -247,14 +254,16 @@
# Jan 1 - Jan 15 and Dec 16 - Dec 31.
lts_concat = np.concatenate([[lts[-1]], lts, [lts[0]]])

time_utc = tools._pandas_to_utc(time)

# handle leap years
try:
isleap = time.is_leap_year
isleap = time_utc.is_leap_year
except AttributeError:
year = time.year
year = time_utc.year

Check warning on line 263 in pvlib/clearsky.py

View check run for this annotation

Codecov / codecov/patch

pvlib/clearsky.py#L263

Added line #L263 was not covered by tests
isleap = _is_leap_year(year)

dayofyear = time.dayofyear
dayofyear = time_utc.dayofyear
days_leap = _calendar_month_middles(2016)
days_no_leap = _calendar_month_middles(2015)

Expand Down
2 changes: 1 addition & 1 deletion pvlib/iotools/solrad.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,9 @@ def get_solrad(station, start, end,
# Use pd.to_datetime so that strings (e.g. '2021-01-01') are accepted
start = pd.to_datetime(start)
end = pd.to_datetime(end)
dates = pd.date_range(start.floor('d'), end, freq='d')

# Generate list of filenames
dates = pd.date_range(start.floor('d'), end, freq='d')
yhkee0404 marked this conversation as resolved.
Show resolved Hide resolved
station = station.lower()
filenames = [
f"{station}/{d.year}/{station}{d.strftime('%y')}{d.dayofyear:03}.dat"
Expand Down
Loading
Loading