Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R.I.P. Nitter 🪦😭 (...unless?) #919

Closed
devgaucho opened this issue Jun 30, 2023 · 510 comments
Closed

R.I.P. Nitter 🪦😭 (...unless?) #919

devgaucho opened this issue Jun 30, 2023 · 510 comments
Labels
bug Something isn't working discussion Discussion topic help wanted Extra attention is needed

Comments

@devgaucho
Copy link

devgaucho commented Jun 30, 2023

https://techcrunch.com/2023/06/30/twitter-now-requires-an-account-to-view-tweets/

the nitter crawler will need to be recreated...

@animegrafmays
Copy link

it was working this morning but it appears nitter is now broken. viewing any profile results in the following error being thrown

An error has occured in one of your routes.

Detail: jsony.nim(45) parseHook
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
asyncfutures.nim(389) read
Expected { but end reached. At offset: 0

@thechesu
Copy link

Libreddit gone tomorrow.

Wonder how long Invidious and FreeTube will last?

@cyberseihis
Copy link

It's so over

@devgaucho
Copy link
Author

devgaucho commented Jun 30, 2023

the twitter rank has already dropped 6 places from yesterday to today 🤦‍♂️ https://ciscorank.com/s/twitter.com
https://i.imgur.com/e5U9Kox.png

@ericpa06
Copy link

ericpa06 commented Jun 30, 2023

Libreddit gone tomorrow.

Wonder how long Invidious and FreeTube will last?

At least reddit still has the old.reddit.com. Which is a pretty descent front-end (as long as you don't care with privacy), with twitter, you are pretty much forced to use that horrible garbage interface. But yeah, this data rush pretty much will make all platforms to garden walled themselves.

@sunlightspectre
Copy link

There are plenty of people who only engage in the modern internet through front-ends; myself being one of them. I have no interest in registering for sites; it took me ages to register a proper git.

One wonders what will happen when these self-inflicted stab-wounds deplete 'profit' rather then generate it?..
Removing access isn't going to make people want to use a site more.
And it certainly won't inflate user numbers, which I'd imagine is the end goal.

C'est la vie.

I'm glad I got to experience the internet as it was.

@animegrafmays
Copy link

thoughts?


a02ae1feb7da5958bf7ae451012fad1ab34a974910511745bdcf80f674b8d452

@codybrom
Copy link

No chance in hell this is temporary unless it affects ad revenue

@orbea
Copy link

orbea commented Jun 30, 2023

Maybe people will be smart enough to stop using proprietary garbage finally...

@ericpa06
Copy link

"Temporary emergency" measure that will always be in place forever.

@DuplicantWood
Copy link

This fork is working

https://github.com/PrivacyDevel/nitter/

@roryyamm
Copy link

This fork is working

https://github.com/PrivacyDevel/nitter/

But privacydev.net isn't.

@animegrafmays
Copy link

I can confirm after building that fork that it also has the same issue

ERROR jsony.nim(45)            parseHook
asyncfutures.nim(389)    read
redpool.nim(68)          getUserIdIter
asyncfutures.nim(389)    read
asyncfutures.nim(389)    read
asyncfutures.nim(389)    read
asyncfutures.nim(389)    read
asyncfutures.nim(389)    read
Expected { but end reached. At offset: 0
404 Not Found:  --- url: https://api.twitter.com/graphql/gkjsKepM6gl_HmFWoWKfgg/SearchTimeline?variables=%7B%22rawQuery%22%3A%22include%3Anativeretweets+%5C%22%23RoshamboCoin%5C%22%22%2C%22count%22%3A20%2C%22product%22%3A%22Latest%22%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%7D&features=%7B%22blue_business_profile_image_shape_enabled%22%3Afalse%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22freedom_of_speech_not_reach_fetch_enabled%22%3Afalse%2C%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Afalse%2C%22highlights_tweets_tab_ui_enabled%22%3Afalse%2C%22interactive_text_enabled%22%3Afalse%2C%22longform_notetweets_consumption_enabled%22%3Atrue%2C%22longform_notetweets_inline_media_enabled%22%3Afalse%2C%22longform_notetweets_richtext_consumption_enabled%22%3Atrue%2C%22longform_notetweets_rich_text_read_enabled%22%3Afalse%2C%22responsive_web_edit_tweet_api_enabled%22%3Afalse%2C%22responsive_web_enhance_cards_enabled%22%3Afalse%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Afalse%2C%22responsive_web_text_conversations_enabled%22%3Afalse%2C%22responsive_web_twitter_blue_verified_badge_is_enabled%22%3Atrue%2C%22rweb_lists_timeline_redesign_enabled%22%3Atrue%2C%22spaces_2022_h2_clipping%22%3Atrue%2C%22spaces_2022_h2_spaces_communities%22%3Atrue%2C%22standardized_nudges_misinfo%22%3Afalse%2C%22tweet_awards_web_tipping_enabled%22%3Afalse%2C%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Afalse%2C%22tweetypie_unmention_optimization_enabled%22%3Afalse%2C%22verified_phone_label_enabled%22%3Afalse%2C%22vibe_api_enabled%22%3Afalse%2C%22view_counts_everywhere_api_enabled%22%3Afalse%7D

@eylles
Copy link

eylles commented Jun 30, 2023

This fork is working
https://github.com/PrivacyDevel/nitter/

But privacydev.net isn't.

That instance is always pegged from the sheer volume of traffic alone.

@tghe-retford
Copy link

thoughts?

"There is nothing more permanent than a temporary measure". Either now or in the future.

@KOLANICH

This comment was marked as spam.

@bugzillus

This comment was marked as outdated.

@HEPOSHEIKKI
Copy link

This is a sad year for social media.

Cheers lads, was a wild ride.

@notune
Copy link

notune commented Jun 30, 2023

Couldn't we use accounts to scrape the data (one per instance)? As far as I know this was already done for NSFW-content so maybe it could be applied to all content now

@hyzchip
Copy link

hyzchip commented Jun 30, 2023

There is one instance that is still functional:
https://tweet.whateveritworks.org
Partly functional, anyway...individual tweets are displayed, but profiles produce a 502 error.

@pozirk
Copy link

pozirk commented Jun 30, 2023

Well, big thanks to @zedeus for nitter, have been enjoying it for some time.
Hope it can be fixed, but... it was going downhill for some time already.
Around a month ago twitter stopped working with disabled cookies, now this.

@NovemberOrWhatever
Copy link

Okay, well, to Mastodon I guess. It's a shame they're killing this. I don't think there are many here who will respond to this by going back to regular Twitter. I'm now just done with Twitter for good I suppose. Thanks to everyone who tried their best.

@AlyoshaVasilieva
Copy link

Couldn't we use accounts to scrape the data (one per instance)? As far as I know this was already done for NSFW-content so maybe it could be applied to all content now

Almost certainly yes, but 1) this requires registering burner accounts, possibly with some regularity as I suspect they'll get banned, and 2) the accounts will need to be linked to phone numbers because Twitter requires this if your account looks suspicious at all.

I don't think this is all that hard but it adds extra cost on top of running a server and it's annoying.

@oelmekki
Copy link

oelmekki commented Jun 30, 2023

Thanks Nitter devs, you have been heroes of privacy, for a while. :)

I don't think this will affect me much, because I have not been interested in Twitter's content for a while. This network has been slowly and painfully dying since Musk bought it (and even before that, it became mostly a trolling platform for political people to hate each others), so whatever. Most of my social feeds go through Mastodon and Lemmy, nowadays, anyway. This is the end of an era, what comes next is exciting!

@nukeop
Copy link

nukeop commented Jun 30, 2023

Glad to see all sorts of "social" media dying 😍

@stemid
Copy link

stemid commented Jun 30, 2023

Can you scrape the embedded tweet instead?

https://platform.twitter.com/embed/Tweet.html?id=1674865731136020505

@soapingtime
Copy link

i don't particularly know how nitter scrapes, but i'm able to see tweets and profiles in my browser by setting the useragent to "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

@Trit34
Copy link

Trit34 commented Jun 30, 2023

i don't particularly know how nitter scrapes, but i'm able to see tweets and profiles in my browser by setting the useragent to "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Someone has reported that this trick does not work anymore. Fun fact: it was the same trick that was used to get the pre-Polymer UI back on YouTube, when it was still available.

I don’t know how long this “temporary” measure will last, nor if it will be temporary at all. In the worse scenario, it was a real pleasure to use Nitter, @zedeus ! Let’s stay tuned, but until facts get me wrong, I consider both Nitter and Twitter are dead from now on.

Thank you!

(Now, if you want to do a Nitter clone for Mastodon, to be able to browse it without JS needed, I won’t say no to this)

@notune
Copy link

notune commented Jun 30, 2023

Can you scrape the embedded tweet instead?

https://platform.twitter.com/embed/Tweet.html?id=1674865731136020505

This could definitely work. Problem is that we still have to get the tweets from a specific user.

@12joan
Copy link

12joan commented Jul 26, 2023

I'll once again recommend 12joan/twitter-client if you need a quick Twitter-to-RSS solution until things settle down. It's a little easier to self-host than Nitter if you're using Docker, no configuration required. A few of us have been using it for a couple of weeks now without incident.

@ArchivingToolsForWBM
Copy link

I noticed that sensitive content flagged by the user (not age restricted) results in a "this page doesn't exist", and is flagged as age restricted on https://nitter.privacydev.net/

Example:
https://twitter.com/canaaaaamin1003/status/1671702937272467459
https://nitter.privacydev.net/canaaaaamin1003/status/1671702937272467459

Logged out:
image
Logged in:
image
Nitter:
image

@Trit34
Copy link

Trit34 commented Jul 28, 2023

Today, on the instance I use (ntr.nah.re), “Tweets” tab and RSS feeds work fine again. Until the next break…

@Metabot24
Copy link

This instance works but sensitive content is still restricted. Has there been a solution found for that yet?

@Trit34
Copy link

Trit34 commented Aug 15, 2023

Well, it broke again…
#983

@zedeus zedeus reopened this Aug 15, 2023
@devgaucho
Copy link
Author

devgaucho commented Aug 16, 2023

the search for this fork still worked intermittently at commit 813a71e until 10:00pm on August 16, 2023

sometimes it returns nothing and the rss feed returns an error 502 (internal error), but eventually it works

from what I saw they use a proxy system with cache because old searches are updated, but new searches return as 404

@devgaucho
Copy link
Author

game over? https://twitter.com/nima_owji/status/1691850092385370483

https://i.imgur.com/fEKHeMH.png

@devgaucho
Copy link
Author

Says it's optional and Blue-only for now. We're safe.

I hope https://twitter.com/xDaily/status/1691859235951583630

https://imgur.com/eAOUcQ3.png

via dailydot

@erikcas
Copy link

erikcas commented Aug 17, 2023

the search for this fork still worked intermittently at commit 813a71e until 10:00pm on August 16, 2023

sometimes it returns nothing and the rss feed returns an error 502 (internal error), but eventually it works

from what I saw they use a proxy system with cache because old searches are updated, but new searches return as 404

They use a login based alternative?

See commit d5689f2

@devgaucho
Copy link
Author

musk magick formula = less captcha and more subscription

https://imgur.com/pB89XVj.png

https://twitter.com/elonmusk/status/1691969296543711471

@devgaucho
Copy link
Author

the search for this fork still worked intermittently at commit 813a71e until 10:00pm on August 16, 2023
sometimes it returns nothing and the rss feed returns an error 502 (internal error), but eventually it works
from what I saw they use a proxy system with cache because old searches are updated, but new searches return as 404

They use a login based alternative?

See commit d5689f2

probably. from what I noticed they use a cache in the search system, the current lag is more than 10 hours

@devgaucho
Copy link
Author

devgaucho commented Aug 17, 2023

the search for this fork still worked intermittently at commit 813a71e until 10:00pm on August 16, 2023
sometimes it returns nothing and the rss feed returns an error 502 (internal error), but eventually it works
from what I saw they use a proxy system with cache because old searches are updated, but new searches return as 404

They use a login based alternative?
See commit d5689f2

probably. from what I noticed they use a cache in the search system, the current lag is more than 10 hours

the cache prioritizes some profiles, the most accessed I assume

https://imgur.com/gATHxP2.png

@devgaucho
Copy link
Author

in July 21 they exchanged the endpoint of the profiles for the endpoint of the search system, I believe that this is the main source of the tweets that feed the profiles, so they simplify by focusing efforts on only one endpoint

https://imgur.com/n8TLon4.png

@devgaucho
Copy link
Author

devgaucho commented Aug 17, 2023

as some delay but stil works

https://imgur.com/HREtJA5.png

@michaelskyba
Copy link

michaelskyba commented Aug 17, 2023

as some delay but stil works

Which accounts are working for you besides "jack" and "elonmusk"?
I tried a few from https://en.wikipedia.org/wiki/List_of_most-followed_Twitter_accounts (like BarackObama, justinbieber, Cristiano, rihanna, YouTube) and all of them showed the same "An error has occured in one of your routes." page. All of them are visible without an account on twitter.com.

Or is this a result of the 10-hour cache delay you mentioned?

@devgaucho
Copy link
Author

as some delay but stil works

Which accounts are working for you besides "jack" and "elonmusk"? I tried a few from https://en.wikipedia.org/wiki/List_of_most-followed_Twitter_accounts (like BarackObama, justinbieber, Cristiano, rihanna, YouTube) and all of them showed the same "An error has occured in one of your routes." page. All of them are visible without an account on twitter.com.

Or is this a result of the 10-hour cache delay you mentioned?

my personal account is accessible thourght the search box with a 2 hours delay

@devgaucho
Copy link
Author

as some delay but stil works

Which accounts are working for you besides "jack" and "elonmusk"? I tried a few from https://en.wikipedia.org/wiki/List_of_most-followed_Twitter_accounts (like BarackObama, justinbieber, Cristiano, rihanna, YouTube) and all of them showed the same "An error has occured in one of your routes." page. All of them are visible without an account on twitter.com.

Or is this a result of the 10-hour cache delay you mentioned?

the cache is local, should prioritize the most accessed sites in the instance in case

@coda101
Copy link

coda101 commented Aug 17, 2023

All of them are visible without an account on twitter.com.

The ones I checked all had a mix of random new and old tweets almost like "highlights".

@unseenlarks
Copy link

apparently, tweets with more likes are being shown first (when viewing a profile when logged out).

https://9to5google.com/2023/08/16/twitter-profile-sort-like-counts-logged-out/

@zedeus
Copy link
Owner

zedeus commented Aug 18, 2023

Superseded by #983

@zedeus zedeus closed this as completed Aug 18, 2023
@rohitrsm83
Copy link

Hi,

For now the only instance that seems to be working is nitter.net. I usually use unofficialbird but that one--and many others--report the jester route exception error.

@zedeus
Copy link
Owner

zedeus commented Aug 21, 2023

@rohitrsm83 the conversation continues here: #983

@zedeus zedeus unpinned this issue Aug 21, 2023
@rohitrsm83
Copy link

@rohitrsm83 the conversation continues here: #983

Thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working discussion Discussion topic help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests