bgpstream_record_get_next_elem being sucked and not giving any output #104
Comments
|
@ksalamat can you provide the version of the libbgpstream you're using and the environment you code is running on? If possible, can you provide a code snippet so that we can reproduce the issue? |
|
Hi
Many thanks.
Using libbgpstream V2 beta.
I code in C++ on a mac.
Here is a code snippet.
I am using all RIPE collectors (from rrc0 to rrc21.
t1= 1574127600 for example and the dumpDuration can be 8*60*60 (8 hours).
With these values it get stucked.
Thanls
Kv
bs = bgpstream_create();
for (auto const &collector : collectors) {
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_COLLECTOR, collector.first.c_str());
}
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_RECORD_TYPE, "updates");
bgpstream_add_interval_filter(bs, t1, t1+interval*dumpDuration-1);
bgpstream_start(bs);
/* Read the stream of records */
while (bgpstream_get_next_record(bs, &record) > 0) {
/* Ignore invalid records */
if (record->status != BGPSTREAM_RECORD_STATUS_VALID_RECORD) {
continue;
}
// std::string collector(record->attributes.dump_collector);
collector = string{record->collector_name};
time = record->time_sec;
/* Extract elems from the current record */
// while ((elem = bgpstream_record_get_next_elem(record)) != NULL) {
while (bgpstream_record_get_next_elem(record, &elem)>0) {
proceed = false;
if (elem->type == BGPSTREAM_ELEM_TYPE_ANNOUNCEMENT ||
elem->type == BGPSTREAM_ELEM_TYPE_WITHDRAWAL) {
if (elem->prefix.address.version == BGPSTREAM_ADDR_VERSION_IPV6) {
if ((version==6) || (version==64)){
proceed=true;
}
} else {
if ((version==4) || (version==64)){
proceed=true;
}
}
}
if (proceed) {
count++;
if (count % 1000000 == 0) {
std::cout<<count<<std::endl;
std::cout<<cache->bgpEvents.size()<<std::endl;
}
bgpMessage = bgpMessagePool->getBGPMessage(order++, elem, time, collector);
if (bgpMessage != NULL){
fifoQueue.add(bgpMessage);
}
}
}
}
bgpstream_destroy(bs);
… Le 22 mars 2020 à 23:35, Mingwei Zhang ***@***.***> a écrit :
@ksalamat <https://github.com/ksalamat> can you provide the version of the libbgpstream you're using and the environment you code is running on? If possible, can you provide a code snippet so that we can reproduce the issue?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXHHTOKJ5QPVTSIK4F7LZLRI2HBRANCNFSM4LQ3TMSQ>.
|
|
Hi Mingwei,
Hope that you are doing well. I have still the issue. I have been able to pinpoint where it happens precisely:
if (bgpstream_reader_open_wait(el->reader) != 0) {
return -1;
}
The code get stuccoed at the the bgpstream_reader_open_wait(el->reader) in line 543 and in function sort_res_list.
Have you an idea what is generating the issue ?
Many thanks
Kv
… Le 22 mars 2020 à 23:48, Kave Salamatian ***@***.***> a écrit :
Hi
Many thanks.
Using libbgpstream V2 beta.
I code in C++ on a mac.
Here is a code snippet.
I am using all RIPE collectors (from rrc0 to rrc21.
t1= 1574127600 for example and the dumpDuration can be 8*60*60 (8 hours).
With these values it get stucked.
Thanls
Kv
bs = bgpstream_create();
for (auto const &collector : collectors) {
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_COLLECTOR, collector.first.c_str());
}
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_RECORD_TYPE, "updates");
bgpstream_add_interval_filter(bs, t1, t1+interval*dumpDuration-1);
bgpstream_start(bs);
/* Read the stream of records */
while (bgpstream_get_next_record(bs, &record) > 0) {
/* Ignore invalid records */
if (record->status != BGPSTREAM_RECORD_STATUS_VALID_RECORD) {
continue;
}
// std::string collector(record->attributes.dump_collector);
collector = string{record->collector_name};
time = record->time_sec;
/* Extract elems from the current record */
// while ((elem = bgpstream_record_get_next_elem(record)) != NULL) {
while (bgpstream_record_get_next_elem(record, &elem)>0) {
proceed = false;
if (elem->type == BGPSTREAM_ELEM_TYPE_ANNOUNCEMENT ||
elem->type == BGPSTREAM_ELEM_TYPE_WITHDRAWAL) {
if (elem->prefix.address.version == BGPSTREAM_ADDR_VERSION_IPV6) {
if ((version==6) || (version==64)){
proceed=true;
}
} else {
if ((version==4) || (version==64)){
proceed=true;
}
}
}
if (proceed) {
count++;
if (count % 1000000 == 0) {
std::cout<<count<<std::endl;
std::cout<<cache->bgpEvents.size()<<std::endl;
}
bgpMessage = bgpMessagePool->getBGPMessage(order++, elem, time, collector);
if (bgpMessage != NULL){
fifoQueue.add(bgpMessage);
}
}
}
}
bgpstream_destroy(bs);
> Le 22 mars 2020 à 23:35, Mingwei Zhang ***@***.*** ***@***.***>> a écrit :
>
>
> @ksalamat <https://github.com/ksalamat> can you provide the version of the libbgpstream you're using and the environment you code is running on? If possible, can you provide a code snippet so that we can reproduce the issue?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXHHTOKJ5QPVTSIK4F7LZLRI2HBRANCNFSM4LQ3TMSQ>.
>
|
|
Hi Mingwei,
Hope that you are doing well. I have still the issue. I have been able to pinpoint where it happens precisely:
if (bgpstream_reader_open_wait(el->reader) != 0) {
return -1;
}
The code get stuccoed at the the bgpstream_reader_open_wait(el->reader) in line 543 and in function sort_res_list.
Have you an idea what is generating the issue ?
Many thanks
Kv
… Le 22 mars 2020 à 23:48, Kave Salamatian ***@***.*** ***@***.***>> a écrit :
Hi
Many thanks.
Using libbgpstream V2 beta.
I code in C++ on a mac.
Here is a code snippet.
I am using all RIPE collectors (from rrc0 to rrc21.
t1= 1574127600 for example and the dumpDuration can be 8*60*60 (8 hours).
With these values it get stucked.
Thanls
Kv
bs = bgpstream_create();
for (auto const &collector : collectors) {
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_COLLECTOR, collector.first.c_str());
}
bgpstream_add_filter(bs, BGPSTREAM_FILTER_TYPE_RECORD_TYPE, "updates");
bgpstream_add_interval_filter(bs, t1, t1+interval*dumpDuration-1);
bgpstream_start(bs);
/* Read the stream of records */
while (bgpstream_get_next_record(bs, &record) > 0) {
/* Ignore invalid records */
if (record->status != BGPSTREAM_RECORD_STATUS_VALID_RECORD) {
continue;
}
// std::string collector(record->attributes.dump_collector);
collector = string{record->collector_name};
time = record->time_sec;
/* Extract elems from the current record */
// while ((elem = bgpstream_record_get_next_elem(record)) != NULL) {
while (bgpstream_record_get_next_elem(record, &elem)>0) {
proceed = false;
if (elem->type == BGPSTREAM_ELEM_TYPE_ANNOUNCEMENT ||
elem->type == BGPSTREAM_ELEM_TYPE_WITHDRAWAL) {
if (elem->prefix.address.version == BGPSTREAM_ADDR_VERSION_IPV6) {
if ((version==6) || (version==64)){
proceed=true;
}
} else {
if ((version==4) || (version==64)){
proceed=true;
}
}
}
if (proceed) {
count++;
if (count % 1000000 == 0) {
std::cout<<count<<std::endl;
std::cout<<cache->bgpEvents.size()<<std::endl;
}
bgpMessage = bgpMessagePool->getBGPMessage(order++, elem, time, collector);
if (bgpMessage != NULL){
fifoQueue.add(bgpMessage);
}
}
}
}
bgpstream_destroy(bs);
> Le 22 mars 2020 à 23:35, Mingwei Zhang ***@***.*** ***@***.***>> a écrit :
>
>
> @ksalamat <https://github.com/ksalamat> can you provide the version of the libbgpstream you're using and the environment you code is running on? If possible, can you provide a code snippet so that we can reproduce the issue?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <#104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXHHTOKJ5QPVTSIK4F7LZLRI2HBRANCNFSM4LQ3TMSQ>.
>
|
|
Hi @ksalamat , |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Alistair,
thanks for the hard work on libbgpstream. I have an issue that is bugging me for a long time.
I want to gather bgp data on long time frame (like 15 days in past). When I run my program everything goes fine for some hours of data (sometime 4 hours of BGP, sometimes even more). But at some (random) point of time bgpstream_record_get_next_elem get blocked and does not return like if it is not receiving data anymore. I have a watchdog timer that is set to 10 min. So the bgpstream_record_get_next_elem can get sucked for more than 10 mins. I am wondering what to do.
One solution would be for me to detect such an event through my watchdog and remake my bgpstream, but as bgpstream_record_get_next_elem is not yielding back I have to kill my thread and rerun it. So would be nice to have a timer inside bgpstream_record_get_next_elem that yield back with an eventual error code.
A second solution would be to figure out why such blocking can happen. Any idea is welcome.
Rgds
The text was updated successfully, but these errors were encountered: