Dear Astro-COLIBRI Team - first of all thanks for your hard work!
On the web-interface it is quite easy to get a list of all (e.g.) GRB events found during a set timeframe - I would love to get this list from the API aswell, for further processing. Mid-december I figured I could do this using the cone-search endpoint, and it worked quite well, just setting the radius to a high number. Now, however, I get timeouts when using a radius > ca. 20. Is there a way to circumvent that and still get my required all-sky list?
The code Iāve been using is exactly from the notebook under āUse the cone search endpoint with custom filtersā and setting radius to e.g. 100.
Thank you in advance!
Hi!
Thanks a lot for reporting this!
We can indeed reproduce your error. At first sight it looks like it is due to an inconsistency in one of our databases that cause the request to fail. Weāll have a closer look (and hopefully a solution) soon.
Btw. we are also already working on implementing a dedicated endpoint for these searches. It will most likely be called /latest_transients and will allow to retrieve the full list of transients fitting a given filter settings in a given time window. This will be accessible from the API but also directly from our interfaces. It is schedule to become available with the next major release.
1 Like
The release v2.9.0 (January 15, 2024) introduces the latest_transients endpoint that can be used from the API but also from the graphical interfaces to download selected events. Note that timeouts can still occur for large requests (i.e. covering very long time ranges) due to limited bandwith to our underlying database.
2 Likes
Hi all!
the latest_transients endpoint documentation refers to āThe filter structure (see dedicated documentation)ā . Where can i find the documentation?
Is it possible possible to refer to my profile filtering in the request?
Thank you in advance!
Thanks a lot for the feedback!
Indeed, the statement is not very clear and weāll update it. What it should point to is the interactive notebook available on Google Colab
This should give you an introduction to the definition of the event filters. Note that you can download the notebook as jupyiter notebook, copy it to your own GoogleColab space, etc. in order to retain the changes you may want to do to it.
With āprofile filteringā you probably mean the filters set up in the graphical interfaces? Two point related to that:
- At this stage you can not use your settings in the interface in a request to the API. But you could use the interfaces directly to download the events fullfilling your selection criteria. There is a ādownloadā button in the top row of the web-interface and at the bottom of the filtering menu in the app:
![Capture dāeĢcran 2024-03-02 aĢ 10.34.27](https://canada1.discourse-cdn.com/flex007/uploads/astro_colibri/original/1X/9996f32eae4b8f14448e2f394e9b3245666f1c6c.png)
- Let me use this opportunity to report on a known issue with the filter settings: they are currently not stored correctly in your user account. This means that the default filters are re-activated when reloading the web-interface and that the synchronization between the web and app interfaces is not working as expected. We are working towards a solutionā¦
1 Like
Hi all; Iām writing a desktop program (with Delphi) to download selected events.
Therefore, Iām setting up a Json Filter to collect events (according to ātypeā i,e. SN,OT,Unclassified OT).
Assuming that it is possible with a spec āfieldspecificationā , What should be the related āfieldā and āvalueā keywords ?
Any extended documentation about the possibilities of the spec āfieldspecificationā ?
Thanks.
1 Like
Hi Jean-Paul,
The keyword field
refers to the fieldās name in the database on which to apply the filter. We do not yet have detailed documentation on this point, but as it is the same as the keyword used in the JSON returned, everything can be inferred from the API endpointās answer.
The keyword value
refers to the value to which the data will be compared. The exact impact will depend on the operation chosen (with the keyword operation
).
As example :
-
In this first example, the system will compare the data stored in the DB field observatory
and keep only entries with data identical to the given value
, aka icecube.
{ 'type': 'FieldSpecification', 'field': 'observatory', 'value': 'icecube', 'operation': '==', 'typeField': 'string' }
-
In this second example, the system will compare the value stored in the DB field magnitude
and keep the one below the value of 18.
{ 'type': 'FieldSpecification', 'field': 'magnitude', 'value': 18, 'operation': '<', 'typeField': 'float' }
I hope it is clearer now; otherwise, let me know. I could help you to construct the first requests.
1 Like
Hi Mathieu,
Iām still exploring some issues with /latest_transientsā¦
the system will compare the value stored in the DB field magnitude
and keep the one below the value of 18. The field āmagnitudeā does not appears in the VoEvents output; Using the notebook, changing the magnitude limit value has no effect on number of results returned (391).
Concerning the type of events, the documentation page is not strict about the desination to use for a query SN SNe OT OTs.
Thanks for your precious help.
JPG
Hi Jean-Paul,
Thanks for your feedback. Iāve noticed there were a mistake in the notebook. The magnitude
field should have been named transient_flux
. For optical transients, the value correspond the magnitude at detection. If you look at the notebook you should see the correct version now, sorry for the confusion caused by the mistake.
You could found below a filter to gather all supernovae, with a magnitude below 18.
my_filters = {
'type': 'AndSpecification',
'specs': [
{
'type': 'FieldSpecification',
'field': 'transient_flux',
'value': 18,
'operation': '<',
'typeField': 'float'
},
{
'type': 'FieldSpecification',
'field': 'type',
'value': 'ot_sn',
'operation': '==',
'typeField': 'string'
}
],
}
If you are looking for a specific type of supernovae, you could look at the classification
field in the returned data and add a filter on this field.
If you are looking at different kind of optical transient, instead for type
to search the value ot_sn
, you could keep the values ot_other
for all the other types of classified transients or ot
for all the non classified transients.
2 Likes
Hi Mathieu,
I have currently problems with my filter which stopped working with āerror 400 Bad requestā.
I tried with yours (perhaps outof date) and got the same message.
Do we have to change anything with the new version?
Thanks for your follow up.
1 Like
Hi Jean-Paul,
thanks for reporting this!
There was indeed a bug in the latest version that prevented queries like yours. I just launched a new instance of the API that should fix this issue. Can you give it another try and let us know?
Thanks Fabian. Itās now running fine.
1 Like
Hi Fabian,
Return of the problem with āerror 400 Bad requestā .
How to run the API with the filters selected and kept for my profile?
That is very strange as nothing has changed (at first sight) on our end. Can you send me the filter settings you are using so that we can run some tests?
We havenāt (yet) a way to use the filters stored in the user profiles for API calls. But this is something weāll be working on soon.
Hi Fabian,
Sorry for this false alert: using an invalid module late in the night.
1 Like