Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3653 - extract username correctly when there is a query #385

Merged
merged 3 commits into from
Sep 6, 2023
Merged

3653 - extract username correctly when there is a query #385

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
200a9fd
make regex more robust
vasconsaurus Sep 5, 2023
6e0067d
fix regex
vasconsaurus Sep 5, 2023
38cfe44
fix regex to be less broad
vasconsaurus Sep 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions app/models/parser/twitter_profile.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ def type

def patterns
[
/^https?:\/\/(www\.)?twitter\.com\/(?<username>[\w\d]+)(\?+.*)$/,
/^https?:\/\/(0|m|mobile)\.twitter\.com\/(?<username>[\w\d]+)(\?+.*)$/,
/^https?:\/\/(www\.)?twitter\.com\/(?<username>[^\/]+)$/,
/^https?:\/\/(0|m|mobile)\.twitter\.com\/(?<username>[^\/]+)$/
Comment on lines 14 to 15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: These two regular expressions are still too broad and will keep matching things we don't want, right? But it can be a separate ticket, where we follow this: https://help.twitter.com/en/managing-your-account/twitter-username-rules#error

Your username cannot be longer than 15 characters. Your name can be longer (50 characters) or shorter than 4 characters, but usernames are kept shorter for the sake of ease.

A username can only contain alphanumeric characters (letters A-Z, numbers 0-9) with the exception of underscores, as noted above. Check to make sure your desired username doesn't contain any symbols, dashes, or spaces.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I kept it as it was from our previous twitter parser (that used v1.1). Anything that has twitter and isn't a tweet will match this one. Do we want to keep it this broad or make it more specific? If someone mistyped, e.g: twitter.com/fake user, it would still try to parse it as a twitter link (though, at this moment, it would fail)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave it as is for now, and refine as we notice those cases (like we're handling in this ticket).

]
Expand Down
15 changes: 15 additions & 0 deletions test/models/parser/twitter_profile_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,19 @@ def stub_profile_lookup
match_one = Parser::TwitterProfile.match?('https://twitter.com/meedan')
assert_equal true, match_one.is_a?(Parser::TwitterProfile)

# Profile with query
match_one = Parser::TwitterProfile.match?('https://twitter.com/meedan?ref_src=twsrc%5Etfw')
assert_equal true, match_one.is_a?(Parser::TwitterProfile)

# Mobile patterns
match_two = Parser::TwitterProfile.match?('https://0.twitter.com/meedan')
assert_equal true, match_two.is_a?(Parser::TwitterProfile)
match_three = Parser::TwitterProfile.match?('https://m.twitter.com/meedan')
assert_equal true, match_three.is_a?(Parser::TwitterProfile)
match_four = Parser::TwitterProfile.match?('https://mobile.twitter.com/meedan')
assert_equal true, match_four.is_a?(Parser::TwitterProfile)
match_five = Parser::TwitterProfile.match?('https://mobile.twitter.com/meedan?ref_src=twsrc%5Etfw')
Dismissed Show dismissed Hide dismissed
assert_equal true, match_five.is_a?(Parser::TwitterProfile)
end

test "it makes a get request to the user lookup by username endpoint successfully" do
Expand Down Expand Up @@ -193,4 +199,13 @@ def stub_profile_lookup
oembed_url = Parser::TwitterProfile.new('https://twitter.com/fake-account').oembed_url
assert_equal 'https://publish.twitter.com/oembed?url=https://twitter.com/fake-account', oembed_url
end

test "should parse tweet profile with a query on the url" do
stub_profile_lookup.returns(twitter_profile_response_success)

data = Parser::TwitterProfile.new('https://www.twitter.com/fake_user?ref_src=twsrc%5Etfw').parse_data(empty_doc)

assert_equal 'fake_user', data['external_id']
assert_equal '@fake_user', data['username']
end
Comment on lines +203 to +210
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice test!

end
Loading