Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parser for twitter search URLs #420

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/models/media.rb
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ def as_json(options = {})
PARSERS = [
Parser::YoutubeProfile,
Parser::YoutubeItem,
Parser::TwitterSearchItem,
Parser::TwitterProfile,
Parser::TwitterItem,
Parser::FacebookProfile,
Expand Down
40 changes: 40 additions & 0 deletions app/models/parser/twitter_search_item.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
module Parser
class TwitterSearchItem < Base
include ProviderTwitter

TWITTER_SEARCH_ITEM_URL = /^https?:\/\/((0|m|mobile|www)\.)?twitter\.com\/search\?(?:[^&\s"]+&)*q=[^&\s"]+/

class << self
def type
'twitter_search_item'.freeze
end

def patterns
[TWITTER_SEARCH_ITEM_URL]
end
end

private

# Main function for class
def parse_data_for_parser(_doc, _original_url, _jsonld_array)
handle_exceptions(StandardError) do
uri = URI.parse @url
params = Hash[URI.decode_www_form uri.query]
src = params['src']
caiosba marked this conversation as resolved.
Show resolved Hide resolved
search_term = params['q']

@parsed_data.merge!(
title: search_term,
search_term: search_term,
src: src,
jayjay-w marked this conversation as resolved.
Show resolved Hide resolved
description: uri,
raw: {
src: src
}
)
end
parsed_data
end
end
end
12 changes: 12 additions & 0 deletions test/models/parser/twitter_item_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ def stub_tweet_lookup
assert_equal true, match_six.is_a?(Parser::TwitterItem)
match_seven = Parser::TwitterItem.match?('http://twitter.com/%23!/salmaeldaly/status/45532711472992256')
assert_equal true, match_seven.is_a?(Parser::TwitterItem)

# Search URLs
match_eight = Parser::TwitterSearchItem.match?('https://twitter.com/search?q=guacamole')
assert_equal true, match_eight.is_a?(Parser::TwitterSearchItem)
end

test "it makes a get request to the tweet lookup endpoint successfully" do
Expand Down Expand Up @@ -197,6 +201,14 @@ def stub_tweet_lookup
assert_match 'Youths! Webb observed galaxy cluster El Gordo', data['title']
end

test "should parse valid search url" do
stub_tweet_lookup.returns(twitter_item_response_success)

data = Parser::TwitterSearchItem.new('https://twitter.com/search?q=ISS%20from:@Space_Station&src=typed_query&f=live').parse_data(empty_doc)

assert_match 'ISS from:@Space_Station', data['title']
end

jayjay-w marked this conversation as resolved.
Show resolved Hide resolved
test "should fill in html when html parsing fails but API works" do
stub_tweet_lookup.returns(twitter_item_response_success)

Expand Down
Loading