From: Anisse Astier <anisse@astier.eu>
Date: Mon, 7 Apr 2014 21:09:53 +0000 (+0200)
Subject: [extractor/common] fallback on utf-8 when charset is not found
X-Git-Url: http://git.bitcoin.ninja/index.cgi?a=commitdiff_plain;h=ec0fafbb1915b830035f621f4660c7e46b1a53ed;hp=a5863bdf331e6a54068912ea216612e812d7100d;p=youtube-dl

[extractor/common] fallback on utf-8 when charset is not found

fixes #2721
---

diff --git a/youtube_dl/extractor/common.py b/youtube_dl/extractor/common.py
index da4193734..9653d44eb 100644
--- a/youtube_dl/extractor/common.py
+++ b/youtube_dl/extractor/common.py
@@ -251,7 +251,10 @@ class InfoExtractor(object):
             with open(filename, 'wb') as outf:
                 outf.write(webpage_bytes)
 
-        content = webpage_bytes.decode(encoding, 'replace')
+        try:
+            content = webpage_bytes.decode(encoding, 'replace')
+        except LookupError:
+            content = webpage_bytes.decode('utf-8', 'replace')
 
         if (u'<title>Access to this site is blocked</title>' in content and
                 u'Websense' in content[:512]):