New module: HiGal catalog + image cutout service #1324

keflavich · 2019-01-03T23:38:24Z

This is a new tool to query the HiGal image cutout and catalog service. The catalog can be accessed from Vizier and therefore is not necessary, but the image cutout service is unique.

The service is hosted at https://tools.ssdc.asi.it/HiGALSearch.jsp. I have never been able to find that URL by googling it.

I still need to make some local tests, but remote tests exist.

keflavich · 2019-01-03T23:40:30Z

TODO:

documentation
local tests

bsipocz · 2019-01-03T23:43:20Z

what other stuff lives at ssdc? Do we foresee that any of those will need a module in the future? If yes, in that case it may make sense to restructure from the beginning and call it ssdc and have higal under it.

keflavich · 2019-01-03T23:52:05Z

Good questions. I have no clue. The sub-interface for HiGal appears to be at least somewhat distinct from the general SSDC querier. We need one of them onboard....

keflavich · 2019-01-04T00:06:56Z

An interesting deficiency I discovered working on this is that astropy.utils.data.get_readable_fileobj does not work for files that require a cookie. That's an oversight we could possibly handle in the distant future.

bsipocz · 2019-01-04T00:10:43Z

what is the status of the other Herschel surveys then? maybe a herschel module then? Though that would be far from any homogeneity.

keflavich · 2019-01-04T00:11:52Z

unfortunately, it looks like Herschel surveys are hosted randomly. Many are at IPAC, others are not.

pep8speaks · 2019-02-28T00:28:30Z

Hello @keflavich! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file astroquery/herschel/higal/core.py:

Line 47:22: E124 closing bracket does not match visual indentation
Line 53:25: E124 closing bracket does not match visual indentation
Line 88:48: E124 closing bracket does not match visual indentation
Line 133:43: E262 inline comment should start with '# '
Line 221:43: E124 closing bracket does not match visual indentation
Line 223:13: E265 block comment should start with '# '
Line 225:13: E265 block comment should start with '# '
Line 245:48: E124 closing bracket does not match visual indentation
Line 262:22: E124 closing bracket does not match visual indentation

In the file astroquery/herschel/higal/tests/test_higal.py:

Line 27:15: E124 closing bracket does not match visual indentation
Line 28:14: E124 closing bracket does not match visual indentation
Line 59:1: E302 expected 2 blank lines, found 1
Line 66:1: E302 expected 2 blank lines, found 1

Comment last updated at 2024-09-21 20:55:30 UTC

bsipocz · 2019-02-28T00:33:45Z

An interesting deficiency I discovered working on this is that astropy.utils.data.get_readable_fileobj does not work for files that require a cookie. That's an oversight we could possibly handle in the distant future.

Apparently I haven't noticed this comment before. Do you suggest that we handle that here, or upstream? If the latter can you open an issue for astropy?

keflavich · 2019-02-28T00:35:59Z

I guess upstream? But how often is this actually needed? I'm not really sure how we'd handle it upstream anyway; requests makes it so much easier I might just say we should require it for cookie-related downloads.

So... not upstream, then!

bsipocz · 2019-02-28T00:55:01Z

preferring to userequests sounds actually super sensible.

kakirastern · 2019-03-18T19:54:48Z

Can I help with this PR to help bring it to fruition (as suggested by @keflavich earlier)? If I am to finish up this PR I might need to open an other PR to "supplant" this...

keflavich · 2019-03-18T20:04:00Z

Yes, go ahead and work on this one if you like. I don't remember where I left it off, but maybe all we need to do is get tests passing?

You can submit a new PR based on this one. Normally I'd prefer to just give write permissions, but I don't know how to do that.

kakirastern · 2019-03-18T20:17:19Z

Yeah, I would prefer to get write permissions too. That would be easier for me as well. Would this help?: https://help.github.com/en/articles/managing-an-individuals-access-to-an-organization-repository

keflavich · 2019-03-18T20:25:56Z

Right, I forgot I can just give you write permissions to my fork - we don't (and can't) give out permission to write to astropy/astroquery. Anyway, you should now have permission to write to this branch.

kakirastern · 2019-03-18T20:28:11Z

No problem, I can add new changes to this PR via your fork... Will follow up on this very soon.

codecov · 2019-03-26T23:13:54Z

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (1c6b7bf) 66.80% compared to head (a56b91b) 66.79%.

Files	Patch %	Lines
astroquery/query.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1324      +/-   ##
==========================================
- Coverage   66.80%   66.79%   -0.01%     
==========================================
  Files         237      237              
  Lines       18320    18321       +1     
==========================================
  Hits        12238    12238              
- Misses       6082     6083       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bsipocz

Some minor things that should fix the failing docs build

bsipocz · 2019-03-27T00:00:31Z

docs/higal/higal.rst

+
+An example catalog query:
+
+.. python::


either remove this (should work without) or add .. code-block:: python

Changed to .. code-block:: python

Added blank line after this line

bsipocz · 2019-03-27T00:00:40Z

docs/higal/higal.rst

+
+And an example image query:
+
+.. python::


Changed to .. code-block:: python

Added blank line after this line

bsipocz · 2019-03-27T00:01:16Z

docs/higal/higal.rst

+Reference/API
+=============
+
+.. automodapi:: astroquery.higal


also link this file in the main index docs.

Added higal/higal.rst to index.rst

bsipocz · 2019-03-27T00:03:01Z

Organizing:should we start it here before it goes in?

@keflavich - would you be against an astroquery.herschel.higal rather than having it in the top namespace?

keflavich · 2019-03-27T00:14:59Z

No, .herschel.higal is fine. It might also end up co-listed under esac.higal and cutout_services.higal eventually?

bsipocz · 2019-03-27T00:26:48Z

esa.higal and cutout_services.higal sounds very good approach to me. The only trick is that we need to make sure users know it's all the same whichever namespace they end up using.

kakirastern · 2019-03-27T06:58:44Z

All checks have passed, ready for the next stage...

bsipocz · 2023-05-08T12:42:48Z

@keflavich - Is this still something on the table? I also wonder whether it should be part of more ssdc functionality (and namespace) rather than to exists on its own?

keflavich · 2023-05-08T12:45:50Z

I keep updating this because I use it in some code for publications, but I've never written a test suite. If SSDC provided other services, I'd be happy to put this tool into that namespace, but I don't know anything else it provides.

So leave this open, I'll keep updating, and someday when I don't have fires to put out, I'll finish this and we can talk namespaces again.

bsipocz · 2023-05-08T12:54:20Z

I pulled up this PR as I'm at a talk about SSDC as is, and it seems that there are more services and tools they provide, many with a VO. But I haven't gone into the weeds of how much of that we could/want/should support.

keflavich · 2023-05-08T12:57:29Z

Oh wow, great! If there are other SSDC products, then, I'm super happy to support them! I hope some of them have nicer APIs - VO would be great.

reminder to me: https://www.ssdc.asi.it/ is the SSDC homepage.

Also, the hack I had before isn't needed when you appropriately query for multiple wavelengths

pep8 lies sometimes.

commit has some fixes

keflavich self-assigned this Jan 3, 2019

keflavich added the New Service label Jan 3, 2019

keflavich changed the title ~~New module: HiGal catalog + image cutuout service~~ New module: HiGal catalog + image cutout service Jan 3, 2019

keflavich mentioned this pull request Feb 25, 2019

Organization? #1215

Open

keflavich force-pushed the higal branch from 0e39716 to 1f1a9bd Compare February 28, 2019 17:21

keflavich added this to the Wishlist milestone Mar 4, 2019

keflavich mentioned this pull request Mar 18, 2019

Herschel Science Archive #395

Closed

bsipocz requested changes Mar 27, 2019

View reviewed changes

bsipocz modified the milestones: Wishlist, v0.3.10 Mar 27, 2019

keflavich force-pushed the higal branch from f1d8a87 to cef03eb Compare January 24, 2021 23:03

keflavich force-pushed the higal branch from cef03eb to b0b7802 Compare May 8, 2021 20:08

bsipocz modified the milestones: v0.4.2, v0.4.3 May 15, 2021

bsipocz modified the milestones: v0.4.3, v0.4.4 Jul 7, 2021

keflavich force-pushed the higal branch from a201529 to 51f5838 Compare July 21, 2021 19:49

bsipocz modified the milestones: v0.4.4, v4 Nov 12, 2021

keflavich force-pushed the higal branch from 51f5838 to c30e2cd Compare February 23, 2022 01:24

keflavich force-pushed the higal branch from 653f214 to ba21c2a Compare January 11, 2024 02:51

keflavich force-pushed the higal branch from ba21c2a to a56b91b Compare February 20, 2024 20:26

keflavich added 13 commits September 21, 2024 16:51

add HiGal module

da77998

more cleanup

8c7db32

add changelog

74f4026

it is possible for a cutout to be missing; we need to allow for that.

4871638

Also, the hack I had before isn't needed when you appropriately query for multiple wavelengths

remove some comments, still not done with the PR

80247e5

minor correction

1f42cec

added basic docs

e1b2b09

pep8 fixes. The bracket closures DO match the indentation, though.

68b7a08

pep8 lies sometimes.

add local tests and cleanup some testing

ceb1273

more interactive tests revealed that my hacks didn't really work, this

66cae7a

commit has some fixes

move everything into astroquery.herschel.higal

e330c20

update relative paths

bb70207

whitespace

42580cb

keflavich force-pushed the higal branch from a56b91b to 42580cb Compare September 21, 2024 20:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New module: HiGal catalog + image cutout service #1324

New module: HiGal catalog + image cutout service #1324

keflavich commented Jan 3, 2019

keflavich commented Jan 3, 2019

bsipocz commented Jan 3, 2019

keflavich commented Jan 3, 2019

keflavich commented Jan 4, 2019

bsipocz commented Jan 4, 2019

keflavich commented Jan 4, 2019

pep8speaks commented Feb 28, 2019 •

edited

Loading

bsipocz commented Feb 28, 2019

keflavich commented Feb 28, 2019

bsipocz commented Feb 28, 2019

kakirastern commented Mar 18, 2019

keflavich commented Mar 18, 2019

kakirastern commented Mar 18, 2019

keflavich commented Mar 18, 2019

kakirastern commented Mar 18, 2019

codecov bot commented Mar 26, 2019 •

edited

Loading

bsipocz left a comment

bsipocz Mar 27, 2019

kakirastern Mar 27, 2019

kakirastern Mar 27, 2019

bsipocz Mar 27, 2019

kakirastern Mar 27, 2019

kakirastern Mar 27, 2019

bsipocz Mar 27, 2019

kakirastern Mar 27, 2019

bsipocz commented Mar 27, 2019

keflavich commented Mar 27, 2019

bsipocz commented Mar 27, 2019

kakirastern commented Mar 27, 2019

bsipocz commented May 8, 2023

keflavich commented May 8, 2023

bsipocz commented May 8, 2023

keflavich commented May 8, 2023

New module: HiGal catalog + image cutout service #1324

Are you sure you want to change the base?

New module: HiGal catalog + image cutout service #1324

Conversation

keflavich commented Jan 3, 2019

keflavich commented Jan 3, 2019

bsipocz commented Jan 3, 2019

keflavich commented Jan 3, 2019

keflavich commented Jan 4, 2019

bsipocz commented Jan 4, 2019

keflavich commented Jan 4, 2019

pep8speaks commented Feb 28, 2019 • edited Loading

Comment last updated at 2024-09-21 20:55:30 UTC

bsipocz commented Feb 28, 2019

keflavich commented Feb 28, 2019

bsipocz commented Feb 28, 2019

kakirastern commented Mar 18, 2019

keflavich commented Mar 18, 2019

kakirastern commented Mar 18, 2019

keflavich commented Mar 18, 2019

kakirastern commented Mar 18, 2019

codecov bot commented Mar 26, 2019 • edited Loading

Codecov Report

bsipocz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bsipocz commented Mar 27, 2019

keflavich commented Mar 27, 2019

bsipocz commented Mar 27, 2019

kakirastern commented Mar 27, 2019

bsipocz commented May 8, 2023

keflavich commented May 8, 2023

bsipocz commented May 8, 2023

keflavich commented May 8, 2023

pep8speaks commented Feb 28, 2019 •

edited

Loading

codecov bot commented Mar 26, 2019 •

edited

Loading