Improve Error Handling and Logging

[thisisayush]

Error Handling in Spiders needs to be improved, here is what's needed to be changed:

• Replace from scrapeNews.pipelines import loggerError with from scrapeNews.settings import logger (loggerError has been removed)
• Replace all loggerError.error("Your Message") with logger.error(__name__ + " Your Message ")
• Wrap All Methods inside a main try..except block

try:
...
except Exception as e:
    logger.error(__name__ + " [UNHANDLED] : " + str(e) + " : " + response.url)

• Use [UNHANDLED] in error message when catching General Errors (i.e catching Exception instead of a particular error)
• Wisely Use self.urls_* variables to reduce inconsitency in stats.

[vipulgupta2048]

• A thing you need to ask yourself before posting issue. Try it sometimes.

1. WHAT (Change logging)
2. WHY (Didn't mention)
3. Where (Easy to figure it out)

• Until you post reasons, hypothesis, opinions about what you are trying to achieve. Will never get across to people. Don't presume that people would know every intention that you have for a particular task that you want to accomplish.

I will let the comminity decide that.

[thisisayush]

WHAT? (Mentioned in title)
Why? (Improvement, mentioned in title)
WHERE? (Described quite well)

Reason being current error handling is very fragile, a single exception can make spider to end which isn't supposed to happen. Read the codes and changes made on sone spiders to see exactly the difference.