---
-### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2016.09.04.1*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
-- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2016.09.04.1**
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2016.09.08*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2016.09.08**
### Before submitting an *issue* make sure you have:
- [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
-[debug] youtube-dl version 2016.09.04.1
+[debug] youtube-dl version 2016.09.08
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
Aleksander Nitecki
Sebastian Blunt
Matěj Cepl
+Xie Yanbo
-version <unreleased>
+version 2016.09.08
Extractors
-+ [bilibili] Support episodes (#10190)
-+ [tvnoe] New extractor (#10524)
++ [jwplatform] Extract height from format label
++ [yahoo] Extract Brightcove Legacy Studio embeds (#9345)
+* [videomore] Fix extraction (#10592)
+* [foxgay] Fix extraction (#10480)
++ [rmcdecouverte] Add extractor for rmcdecouverte.bfmtv.com (#9709)
+* [gamestar] Fix metadata extraction (#10479)
+* [puls4] Fix extraction (#10583)
++ [cctv] Add extractor for CCTV and CNTV (#8153)
++ [lci] Add extractor for lci.fr (#10573)
++ [wat] Extract DASH formats
++ [viafree] Improve video id detection (#10569)
++ [trutv] Add extractor for trutv.com (#10519)
++ [nick] Add support for nickelodeon.nl (#10559)
++ [abcotvs:clips] Add support for clips.abcotvs.com
++ [abcotvs] Add support for ABC Owned Television Stations sites (#9551)
++ [miaopai] Add extractor for miaopai.com (#10556)
+* [gamestar] Fix metadata extraction (#10479)
++ [bilibili] Add support for episodes (#10190)
++ [tvnoe] Add extractor for tvnoe.cz (#10524)
version 2016.09.04.1
youtube-dl --download-archive archive.txt "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re"
+### Should I add `--hls-prefer-native` into my config?
+
+When youtube-dl detects an HLS video, it can download it either with the built-in downloader or ffmpeg. Since many HLS streams are slightly invalid and ffmpeg/youtube-dl each handle some invalid cases better than the other, there is an option to switch the downloader if needed.
+
+When youtube-dl knows that one particular downloader works better for a given website, that downloader will be picked. Otherwise, youtube-dl will pick the best downloader for general compatibility, which at the moment happens to be ffmpeg. This choice may change in future versions of youtube-dl, with improvements of the built-in downloader and/or ffmpeg.
+
+In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](http://rg3.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader.
+
+If you put either `--hls-prefer-native` or `--hls-prefer-ffmpeg` into your configuration, a different subset of videos will fail to download correctly. Instead, it is much better to [file an issue](https://yt-dl.org/bug) or a pull request which details why the native or the ffmpeg HLS downloader is a better choice for your use case.
+
### Can you add support for this anime video site, or site which shows current movies for free?
As a matter of policy (as well as legality), youtube-dl does not include support for services that specialize in infringing copyright. As a rule of thumb, if you cannot easily find a video that the service is quite obviously allowed to distribute (i.e. that has been uploaded by the creator, the creator's distributor, or is published under a free license), the service is probably unfit for inclusion to youtube-dl.
- **9now.com.au**
- **abc.net.au**
- **abc.net.au:iview**
- - **Abc7News**
- **abcnews**
- **abcnews:video**
+ - **abcotvs**: ABC Owned Television Stations
+ - **abcotvs:clips**
- **AcademicEarth:Course**
- **acast**
- **acast:channel**
- **CBSNews**: CBS News
- **CBSNewsLiveVideo**: CBS News Live Videos
- **CBSSports**
+ - **CCTV**
- **CDA**
- **CeskaTelevize**
- **channel9**: Channel 9
- **kuwo:song**: 酷我音乐
- **la7.it**
- **Laola1Tv**
+ - **LCI**
- **Lcp**
- **LcpPlay**
- **Le**: 乐视网
- **Metacritic**
- **Mgoon**
- **MGTV**: 芒果TV
+ - **MiaoPai**
- **Minhateca**
- **MinistryGrid**
- **Minoto**
- **revision3:embed**
- **RICE**
- **RingTV**
+ - **RMCDecouverte**
- **RockstarGames**
- **RoosterTeeth**
- **RottenTomatoes**
- **TrailerAddict** (Currently broken)
- **Trilulilu**
- **trollvids**
+ - **TruTV**
- **Tube8**
- **TubiTv**
- **tudou**
- **TVCArticle**
- **tvigle**: Интернет-телевидение Tvigle.ru
- **tvland.com**
+ - **TVNoe**
- **tvp**: Telewizja Polska
- **tvp:embed**: Telewizja Polska
- **tvp:series**
class AbcNewsVideoIE(AMPIE):
IE_NAME = 'abcnews:video'
- _VALID_URL = 'http://abcnews.go.com/[^/]+/video/(?P<display_id>[0-9a-z-]+)-(?P<id>\d+)'
+ _VALID_URL = r'https?://abcnews\.go\.com/[^/]+/video/(?P<display_id>[0-9a-z-]+)-(?P<id>\d+)'
_TESTS = [{
'url': 'http://abcnews.go.com/ThisWeek/video/week-exclusive-irans-foreign-minister-zarif-20411932',
class AbcNewsIE(InfoExtractor):
IE_NAME = 'abcnews'
- _VALID_URL = 'https?://abcnews\.go\.com/(?:[^/]+/)+(?P<display_id>[0-9a-z-]+)/story\?id=(?P<id>\d+)'
+ _VALID_URL = r'https?://abcnews\.go\.com/(?:[^/]+/)+(?P<display_id>[0-9a-z-]+)/story\?id=(?P<id>\d+)'
_TESTS = [{
'url': 'http://abcnews.go.com/Blotter/News/dramatic-video-rare-death-job-america/story?id=10498713#.UIhwosWHLjY',
class ABCOTVSIE(InfoExtractor):
IE_NAME = 'abcotvs'
+ IE_DESC = 'ABC Owned Television Stations'
_VALID_URL = r'https?://(?:abc(?:7(?:news|ny|chicago)?|11|13|30)|6abc)\.com(?:/[^/]+/(?P<display_id>[^/]+))?/(?P<id>\d+)'
_TESTS = [
{
class ARDIE(InfoExtractor):
- _VALID_URL = '(?P<mainurl>https?://(www\.)?daserste\.de/[^?#]+/videos/(?P<display_id>[^/?#]+)-(?P<id>[0-9]+))\.html'
+ _VALID_URL = r'(?P<mainurl>https?://(www\.)?daserste\.de/[^?#]+/videos/(?P<display_id>[^/?#]+)-(?P<id>[0-9]+))\.html'
_TEST = {
'url': 'http://www.daserste.de/information/reportage-dokumentation/dokus/videos/die-story-im-ersten-mission-unter-falscher-flagge-100.html',
'md5': 'd216c3a86493f9322545e045ddc3eb35',
from __future__ import unicode_literals
+import re
+
from .common import InfoExtractor
from ..utils import float_or_none
class CanvasIE(InfoExtractor):
- _VALID_URL = r'https?://(?:www\.)?canvas\.be/video/(?:[^/]+/)*(?P<id>[^/?#&]+)'
+ _VALID_URL = r'https?://(?:www\.)?(?P<site_id>canvas|een)\.be/(?:[^/]+/)*(?P<id>[^/?#&]+)'
_TESTS = [{
'url': 'http://www.canvas.be/video/de-afspraak/najaar-2015/de-afspraak-veilt-voor-de-warmste-week',
'md5': 'ea838375a547ac787d4064d8c7860a6c',
'params': {
'skip_download': True,
}
+ }, {
+ 'url': 'https://www.een.be/sorry-voor-alles/herbekijk-sorry-voor-alles',
+ 'info_dict': {
+ 'id': 'mz-ast-11a587f8-b921-4266-82e2-0bce3e80d07f',
+ 'display_id': 'herbekijk-sorry-voor-alles',
+ 'ext': 'mp4',
+ 'title': 'Herbekijk Sorry voor alles',
+ 'description': 'md5:8bb2805df8164e5eb95d6a7a29dc0dd3',
+ 'thumbnail': 're:^https?://.*\.jpg$',
+ 'duration': 3788.06,
+ },
+ 'params': {
+ 'skip_download': True,
+ }
+ }, {
+ 'url': 'https://www.canvas.be/check-point/najaar-2016/de-politie-uw-vriend',
+ 'only_matching': True,
}]
def _real_extract(self, url):
- display_id = self._match_id(url)
+ mobj = re.match(self._VALID_URL, url)
+ site_id, display_id = mobj.group('site_id'), mobj.group('id')
webpage = self._download_webpage(url, display_id)
- title = self._search_regex(
+ title = (self._search_regex(
r'<h1[^>]+class="video__body__header__title"[^>]*>(.+?)</h1>',
- webpage, 'title', default=None) or self._og_search_title(webpage)
+ webpage, 'title', default=None) or self._og_search_title(
+ webpage)).strip()
video_id = self._html_search_regex(
r'data-video=(["\'])(?P<id>.+?)\1', webpage, 'video id', group='id')
data = self._download_json(
- 'https://mediazone.vrt.be/api/v1/canvas/assets/%s' % video_id, display_id)
+ 'https://mediazone.vrt.be/api/v1/%s/assets/%s'
+ % (site_id, video_id), display_id)
formats = []
for target in data['targetUrls']:
--- /dev/null
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..utils import float_or_none
+
+
+class CCTVIE(InfoExtractor):
+ _VALID_URL = r'''(?x)https?://(?:.+?\.)?
+ (?:
+ cctv\.(?:com|cn)|
+ cntv\.cn
+ )/
+ (?:
+ video/[^/]+/(?P<id>[0-9a-f]{32})|
+ \d{4}/\d{2}/\d{2}/(?P<display_id>VID[0-9A-Za-z]+)
+ )'''
+ _TESTS = [{
+ 'url': 'http://english.cntv.cn/2016/09/03/VIDEhnkB5y9AgHyIEVphCEz1160903.shtml',
+ 'md5': '819c7b49fc3927d529fb4cd555621823',
+ 'info_dict': {
+ 'id': '454368eb19ad44a1925bf1eb96140a61',
+ 'ext': 'mp4',
+ 'title': 'Portrait of Real Current Life 09/03/2016 Modern Inventors Part 1',
+ }
+ }, {
+ 'url': 'http://tv.cctv.com/2016/09/07/VIDE5C1FnlX5bUywlrjhxXOV160907.shtml',
+ 'only_matching': True,
+ }, {
+ 'url': 'http://tv.cntv.cn/video/C39296/95cfac44cabd3ddc4a9438780a4e5c44',
+ 'only_matching': True
+ }]
+
+ def _real_extract(self, url):
+ video_id, display_id = re.match(self._VALID_URL, url).groups()
+ if not video_id:
+ webpage = self._download_webpage(url, display_id)
+ video_id = self._search_regex(
+ r'(?:fo\.addVariable\("videoCenterId",\s*|guid\s*=\s*)"([0-9a-f]{32})',
+ webpage, 'video_id')
+ api_data = self._download_json(
+ 'http://vdn.apps.cntv.cn/api/getHttpVideoInfo.do?pid=' + video_id, video_id)
+ m3u8_url = re.sub(r'maxbr=\d+&?', '', api_data['hls_url'])
+
+ return {
+ 'id': video_id,
+ 'title': api_data['title'],
+ 'formats': self._extract_m3u8_formats(
+ m3u8_url, video_id, 'mp4', 'm3u8_native', fatal=False),
+ 'duration': float_or_none(api_data.get('video', {}).get('totalLength')),
+ }
class DailymotionCloudIE(DailymotionBaseInfoExtractor):
- _VALID_URL_PREFIX = r'http://api\.dmcloud\.net/(?:player/)?embed/'
+ _VALID_URL_PREFIX = r'https?://api\.dmcloud\.net/(?:player/)?embed/'
_VALID_URL = r'%s[^/]+/(?P<id>[^/?]+)' % _VALID_URL_PREFIX
_VALID_EMBED_URL = r'%s[^/]+/[^\'"]+' % _VALID_URL_PREFIX
)
from .cbssports import CBSSportsIE
from .ccc import CCCIE
+from .cctv import CCTVIE
from .cda import CDAIE
from .ceskatelevize import CeskaTelevizeIE
from .channel9 import Channel9IE
from .karaoketv import KaraoketvIE
from .karrierevideos import KarriereVideosIE
from .keezmovies import KeezMoviesIE
+from .ketnet import KetnetIE
from .khanacademy import KhanAcademyIE
from .kickstarter import KickStarterIE
from .keek import KeekIE
)
from .la7 import LA7IE
from .laola1tv import Laola1TvIE
+from .lci import LCIIE
from .lcp import (
LcpPlayIE,
LcpIE,
from .metacritic import MetacriticIE
from .mgoon import MgoonIE
from .mgtv import MGTVIE
+from .miaopai import MiaoPaiIE
from .microsoftvirtualacademy import (
MicrosoftVirtualAcademyIE,
MicrosoftVirtualAcademyCourseIE,
)
from .rice import RICEIE
from .ringtv import RingTVIE
+from .rmcdecouverte import RMCDecouverteIE
from .ro220 import Ro220IE
from .rockstargames import RockstarGamesIE
from .roosterteeth import RoosterTeethIE
from .telecinco import TelecincoIE
from .telegraaf import TelegraafIE
from .telemb import TeleMBIE
+from .telequebec import TeleQuebecIE
from .teletask import TeleTaskIE
from .telewebion import TelewebionIE
from .testurl import TestURLIE
from __future__ import unicode_literals
+import itertools
+
from .common import InfoExtractor
+from ..utils import (
+ get_element_by_id,
+ remove_end,
+)
class FoxgayIE(InfoExtractor):
_VALID_URL = r'https?://(?:www\.)?foxgay\.com/videos/(?:\S+-)?(?P<id>\d+)\.shtml'
_TEST = {
'url': 'http://foxgay.com/videos/fuck-turkish-style-2582.shtml',
- 'md5': '80d72beab5d04e1655a56ad37afe6841',
+ 'md5': '344558ccfea74d33b7adbce22e577f54',
'info_dict': {
'id': '2582',
'ext': 'mp4',
- 'title': 'md5:6122f7ae0fc6b21ebdf59c5e083ce25a',
- 'description': 'md5:5e51dc4405f1fd315f7927daed2ce5cf',
+ 'title': 'Fuck Turkish-style',
+ 'description': 'md5:6ae2d9486921891efe89231ace13ffdf',
'age_limit': 18,
'thumbnail': 're:https?://.*\.jpg$',
},
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
- title = self._html_search_regex(
- r'<title>(?P<title>.*?)</title>',
- webpage, 'title', fatal=False)
- description = self._html_search_regex(
- r'<div class="ico_desc"><h2>(?P<description>.*?)</h2>',
- webpage, 'description', fatal=False)
+ title = remove_end(self._html_search_regex(
+ r'<title>([^<]+)</title>', webpage, 'title'), ' - Foxgay.com')
+ description = get_element_by_id('inf_tit', webpage)
+ # The default user-agent with foxgay cookies leads to pages without videos
+ self._downloader.cookiejar.clear('.foxgay.com')
# Find the URL for the iFrame which contains the actual video.
+ iframe_url = self._html_search_regex(
+ r'<iframe[^>]+src=([\'"])(?P<url>[^\'"]+)\1', webpage,
+ 'video frame', group='url')
iframe = self._download_webpage(
- self._html_search_regex(r'iframe src="(?P<frame>.*?)"', webpage, 'video frame'),
- video_id)
- video_url = self._html_search_regex(
- r"v_path = '(?P<vid>http://.*?)'", iframe, 'url')
- thumb_url = self._html_search_regex(
- r"t_path = '(?P<thumb>http://.*?)'", iframe, 'thumbnail', fatal=False)
+ iframe_url, video_id, headers={'User-Agent': 'curl/7.50.1'},
+ note='Downloading video frame')
+ video_data = self._parse_json(self._search_regex(
+ r'video_data\s*=\s*([^;]+);', iframe, 'video data'), video_id)
+
+ formats = [{
+ 'url': source,
+ 'height': resolution,
+ } for source, resolution in zip(
+ video_data['sources'], video_data.get('resolutions', itertools.repeat(None)))]
+
+ self._sort_formats(formats)
return {
'id': video_id,
'title': title,
- 'url': video_url,
+ 'formats': formats,
'description': description,
- 'thumbnail': thumb_url,
+ 'thumbnail': video_data.get('act_vid', {}).get('thumb'),
'age_limit': 18,
}
# coding: utf-8
from __future__ import unicode_literals
-import re
-
from .common import InfoExtractor
from ..utils import (
int_or_none,
- parse_duration,
- str_to_int,
- unified_strdate,
+ remove_end,
)
'id': '76110',
'ext': 'mp4',
'title': 'Hobbit 3: Die Schlacht der Fünf Heere - Teaser-Trailer zum dritten Teil',
- 'description': 'Der Teaser-Trailer zu Hobbit 3: Die Schlacht der Fünf Heere zeigt einige Szenen aus dem dritten Teil der Saga und kündigt den vollständigen Trailer an.',
- 'thumbnail': 'http://images.gamestar.de/images/idgwpgsgp/bdb/2494525/600x.jpg',
+ 'description': 'Der Teaser-Trailer zu Hobbit 3: Die Schlacht der Fünf Heere zeigt einige Szenen aus dem dritten Teil der Saga und kündigt den...',
+ 'thumbnail': 're:^https?://.*\.jpg$',
+ 'timestamp': 1406542020,
'upload_date': '20140728',
'duration': 17
}
video_id = self._match_id(url)
webpage = self._download_webpage(url, video_id)
- og_title = self._og_search_title(webpage)
- title = re.sub(r'\s*- Video (bei|-) GameStar\.de$', '', og_title)
-
url = 'http://gamestar.de/_misc/videos/portal/getVideoUrl.cfm?premium=0&videoId=' + video_id
- description = self._og_search_description(webpage).strip()
-
- thumbnail = self._proto_relative_url(
- self._og_search_thumbnail(webpage), scheme='http:')
-
- upload_date = unified_strdate(self._html_search_regex(
- r'<span style="float:left;font-size:11px;">Datum: ([0-9]+\.[0-9]+\.[0-9]+) ',
- webpage, 'upload_date', fatal=False))
-
- duration = parse_duration(self._html_search_regex(
- r' Länge: ([0-9]+:[0-9]+)</span>', webpage, 'duration',
- fatal=False))
-
- view_count = str_to_int(self._html_search_regex(
- r' Zuschauer: ([0-9\.]+) ', webpage,
- 'view_count', fatal=False))
+ # TODO: there are multiple ld+json objects in the webpage,
+ # while _search_json_ld finds only the first one
+ json_ld = self._parse_json(self._search_regex(
+ r'(?s)<script[^>]+type=(["\'])application/ld\+json\1[^>]*>(?P<json_ld>[^<]+VideoObject[^<]+)</script>',
+ webpage, 'JSON-LD', group='json_ld'), video_id)
+ info_dict = self._json_ld(json_ld, video_id)
+ info_dict['title'] = remove_end(info_dict['title'], ' - GameStar')
+ view_count = json_ld.get('interactionCount')
comment_count = int_or_none(self._html_search_regex(
- r'>Kommentieren \(([0-9]+)\)</a>', webpage, 'comment_count',
+ r'([0-9]+) Kommentare</span>', webpage, 'comment_count',
fatal=False))
- return {
+ info_dict.update({
'id': video_id,
- 'title': title,
'url': url,
'ext': 'mp4',
- 'thumbnail': thumbnail,
- 'description': description,
- 'upload_date': upload_date,
- 'duration': duration,
'view_count': view_count,
'comment_count': comment_count
- }
+ })
+
+ return info_dict
class GloboIE(InfoExtractor):
- _VALID_URL = '(?:globo:|https?://.+?\.globo\.com/(?:[^/]+/)*(?:v/(?:[^/]+/)?|videos/))(?P<id>\d{7,})'
+ _VALID_URL = r'(?:globo:|https?://.+?\.globo\.com/(?:[^/]+/)*(?:v/(?:[^/]+/)?|videos/))(?P<id>\d{7,})'
_API_URL_TEMPLATE = 'http://api.globovideos.com/videos/%s/playlist'
_SECURITY_URL_TEMPLATE = 'http://security.video.globo.com/videos/%s/hash?player=flash&version=17.0.0.132&resource_id=%s'
class GloboArticleIE(InfoExtractor):
- _VALID_URL = 'https?://.+?\.globo\.com/(?:[^/]+/)*(?P<id>[^/]+)(?:\.html)?'
+ _VALID_URL = r'https?://.+?\.globo\.com/(?:[^/]+/)*(?P<id>[^/]+)(?:\.html)?'
_VIDEOID_REGEXES = [
r'\bdata-video-id=["\'](\d{7,})',
'ext': ext,
})
else:
+ height = int_or_none(source.get('height'))
+ if height is None:
+ # Often no height is provided but there is a label in
+ # format like 1080p.
+ height = int_or_none(self._search_regex(
+ r'^(\d{3,})[pP]$', source.get('label') or '',
+ 'height', default=None))
a_format = {
'url': source_url,
'width': int_or_none(source.get('width')),
- 'height': int_or_none(source.get('height')),
+ 'height': height,
'ext': ext,
}
if source_url.startswith('rtmp'):
class KaraoketvIE(InfoExtractor):
- _VALID_URL = r'http://www.karaoketv.co.il/[^/]+/(?P<id>\d+)'
+ _VALID_URL = r'https?://www\.karaoketv\.co\.il/[^/]+/(?P<id>\d+)'
_TEST = {
'url': 'http://www.karaoketv.co.il/%D7%A9%D7%99%D7%A8%D7%99_%D7%A7%D7%A8%D7%99%D7%95%D7%A7%D7%99/58356/%D7%90%D7%99%D7%96%D7%95%D7%9F',
'info_dict': {
--- /dev/null
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class KetnetIE(InfoExtractor):
+ _VALID_URL = r'https?://(?:www\.)?ketnet\.be/(?:[^/]+/)*(?P<id>[^/?#&]+)'
+ _TESTS = [{
+ 'url': 'https://www.ketnet.be/kijken/zomerse-filmpjes',
+ 'md5': 'd907f7b1814ef0fa285c0475d9994ed7',
+ 'info_dict': {
+ 'id': 'zomerse-filmpjes',
+ 'ext': 'mp4',
+ 'title': 'Gluur mee op de filmset en op Pennenzakkenrock',
+ 'description': 'Gluur mee met Ghost Rockers op de filmset',
+ 'thumbnail': 're:^https?://.*\.jpg$',
+ }
+ }, {
+ 'url': 'https://www.ketnet.be/kijken/karrewiet/uitzending-8-september-2016',
+ 'only_matching': True,
+ }, {
+ 'url': 'https://www.ketnet.be/achter-de-schermen/sien-repeteert-voor-stars-for-life',
+ 'only_matching': True,
+ }]
+
+ def _real_extract(self, url):
+ video_id = self._match_id(url)
+
+ webpage = self._download_webpage(url, video_id)
+
+ config = self._parse_json(
+ self._search_regex(
+ r'(?s)playerConfig\s*=\s*({.+?})\s*;', webpage,
+ 'player config'),
+ video_id)
+
+ title = config['title']
+
+ formats = self._extract_m3u8_formats(
+ config['source']['hls'], video_id, 'mp4',
+ entry_protocol='m3u8_native', m3u8_id='hls')
+ self._sort_formats(formats)
+
+ return {
+ 'id': video_id,
+ 'title': title,
+ 'description': config.get('description'),
+ 'thumbnail': config.get('image'),
+ 'series': config.get('program'),
+ 'episode': config.get('episode'),
+ 'formats': formats,
+ }
--- /dev/null
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class LCIIE(InfoExtractor):
+ _VALID_URL = r'https?://(?:www\.)?lci\.fr/[^/]+/[\w-]+-(?P<id>\d+)\.html'
+ _TEST = {
+ 'url': 'http://www.lci.fr/international/etats-unis-a-j-62-hillary-clinton-reste-sans-voix-2001679.html',
+ 'md5': '2fdb2538b884d4d695f9bd2bde137e6c',
+ 'info_dict': {
+ 'id': '13244802',
+ 'ext': 'mp4',
+ 'title': 'Hillary Clinton et sa quinte de toux, en plein meeting',
+ 'description': 'md5:a4363e3a960860132f8124b62f4a01c9',
+ }
+ }
+
+ def _real_extract(self, url):
+ video_id = self._match_id(url)
+ webpage = self._download_webpage(url, video_id)
+ wat_id = self._search_regex(r'data-watid=[\'"](\d+)', webpage, 'wat id')
+ return self.url_result('wat:' + wat_id, 'Wat', wat_id)
--- /dev/null
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class MiaoPaiIE(InfoExtractor):
+ _VALID_URL = r'https?://(?:www\.)?miaopai\.com/show/(?P<id>[-A-Za-z0-9~_]+)'
+ _TEST = {
+ 'url': 'http://www.miaopai.com/show/n~0hO7sfV1nBEw4Y29-Hqg__.htm',
+ 'md5': '095ed3f1cd96b821add957bdc29f845b',
+ 'info_dict': {
+ 'id': 'n~0hO7sfV1nBEw4Y29-Hqg__',
+ 'ext': 'mp4',
+ 'title': '西游记音乐会的秒拍视频',
+ 'thumbnail': 're:^https?://.*/n~0hO7sfV1nBEw4Y29-Hqg___m.jpg',
+ }
+ }
+
+ _USER_AGENT_IPAD = 'Mozilla/5.0 (iPad; CPU OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1'
+
+ def _real_extract(self, url):
+ video_id = self._match_id(url)
+ webpage = self._download_webpage(
+ url, video_id, headers={'User-Agent': self._USER_AGENT_IPAD})
+
+ title = self._html_search_regex(
+ r'<title>([^<]+)</title>', webpage, 'title')
+ thumbnail = self._html_search_regex(
+ r'<div[^>]+class=(?P<q1>[\'"]).*\bvideo_img\b.*(?P=q1)[^>]+data-url=(?P<q2>[\'"])(?P<url>[^\'"]+)(?P=q2)',
+ webpage, 'thumbnail', fatal=False, group='url')
+ videos = self._parse_html5_media_entries(url, webpage, video_id)
+ info = videos[0]
+
+ info.update({
+ 'id': video_id,
+ 'title': title,
+ 'thumbnail': thumbnail,
+ })
+ return info
'height': 360,
'duration': 179,
'filesize': 17822500,
- }
+ },
+ 'skip': 'Video has been removed',
},
{
'url': 'http://playreplay.net/video/77107.7f325710a627383d40540d8e991a',
class OnetIE(OnetBaseIE):
- _VALID_URL = 'https?://(?:www\.)?onet\.tv/[a-z]/[a-z]+/(?P<display_id>[0-9a-z-]+)/(?P<id>[0-9a-z]+)'
+ _VALID_URL = r'https?://(?:www\.)?onet\.tv/[a-z]/[a-z]+/(?P<display_id>[0-9a-z-]+)/(?P<id>[0-9a-z]+)'
IE_NAME = 'onet.tv'
_TEST = {
from __future__ import unicode_literals
-import re
-
from .common import InfoExtractor
class ParliamentLiveUKIE(InfoExtractor):
IE_NAME = 'parliamentlive.tv'
IE_DESC = 'UK parliament videos'
- _VALID_URL = r'https?://www\.parliamentlive\.tv/Main/Player\.aspx\?(?:[^&]+&)*?meetingId=(?P<id>[0-9]+)'
+ _VALID_URL = r'https?://(?:www\.)?parliamentlive\.tv/Event/Index/(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
_TEST = {
- 'url': 'http://www.parliamentlive.tv/Main/Player.aspx?meetingId=15121&player=windowsmedia',
+ 'url': 'http://parliamentlive.tv/Event/Index/c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
'info_dict': {
- 'id': '15121',
- 'ext': 'asf',
- 'title': 'hoc home affairs committee, 18 mar 2014.pm',
- 'description': 'md5:033b3acdf83304cd43946b2d5e5798d1',
+ 'id': 'c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
+ 'ext': 'mp4',
+ 'title': 'Home Affairs Committee',
+ 'uploader_id': 'FFMPEG-01',
+ 'timestamp': 1422696664,
+ 'upload_date': '20150131',
},
- 'params': {
- 'skip_download': True, # Requires mplayer (mms)
- }
}
def _real_extract(self, url):
- mobj = re.match(self._VALID_URL, url)
- video_id = mobj.group('id')
- webpage = self._download_webpage(url, video_id)
-
- asx_url = self._html_search_regex(
- r'embed.*?src="([^"]+)" name="MediaPlayer"', webpage,
- 'metadata URL')
- asx = self._download_xml(asx_url, video_id, 'Downloading ASX metadata')
- video_url = asx.find('.//REF').attrib['HREF']
-
- title = self._search_regex(
- r'''(?x)player\.setClipDetails\(
- (?:(?:[0-9]+|"[^"]+"),\s*){2}
- "([^"]+",\s*"[^"]+)"
- ''',
- webpage, 'title').replace('", "', ', ')
- description = self._html_search_regex(
- r'(?s)<span id="MainContentPlaceHolder_CaptionsBlock_WitnessInfo">(.*?)</span>',
- webpage, 'description')
-
+ video_id = self._match_id(url)
+ webpage = self._download_webpage(
+ 'http://vodplayer.parliamentlive.tv/?mid=' + video_id, video_id)
+ widget_config = self._parse_json(self._search_regex(
+ r'kWidgetConfig\s*=\s*({.+});',
+ webpage, 'kaltura widget config'), video_id)
+ kaltura_url = 'kaltura:%s:%s' % (widget_config['wid'][1:], widget_config['entry_id'])
+ event_title = self._download_json(
+ 'http://parliamentlive.tv/Event/GetShareVideo/' + video_id, video_id)['event']['title']
return {
+ '_type': 'url_transparent',
'id': video_id,
- 'ext': 'asf',
- 'url': video_url,
- 'title': title,
- 'description': description,
+ 'title': event_title,
+ 'description': '',
+ 'url': kaltura_url,
+ 'ie_key': 'Kaltura',
}
)
-class ProSiebenSat1IE(InfoExtractor):
+class ProSiebenSat1BaseIE(InfoExtractor):
+ def _extract_video_info(self, url, clip_id):
+ client_location = url
+
+ video = self._download_json(
+ 'http://vas.sim-technik.de/vas/live/v2/videos',
+ clip_id, 'Downloading videos JSON', query={
+ 'access_token': self._TOKEN,
+ 'client_location': client_location,
+ 'client_name': self._CLIENT_NAME,
+ 'ids': clip_id,
+ })[0]
+
+ if video.get('is_protected') is True:
+ raise ExtractorError('This video is DRM protected.', expected=True)
+
+ duration = float_or_none(video.get('duration'))
+ source_ids = [compat_str(source['id']) for source in video['sources']]
+
+ client_id = self._SALT[:2] + sha1(''.join([clip_id, self._SALT, self._TOKEN, client_location, self._SALT, self._CLIENT_NAME]).encode('utf-8')).hexdigest()
+
+ sources = self._download_json(
+ 'http://vas.sim-technik.de/vas/live/v2/videos/%s/sources' % clip_id,
+ clip_id, 'Downloading sources JSON', query={
+ 'access_token': self._TOKEN,
+ 'client_id': client_id,
+ 'client_location': client_location,
+ 'client_name': self._CLIENT_NAME,
+ })
+ server_id = sources['server_id']
+
+ def fix_bitrate(bitrate):
+ bitrate = int_or_none(bitrate)
+ if not bitrate:
+ return None
+ return (bitrate // 1000) if bitrate % 1000 == 0 else bitrate
+
+ formats = []
+ for source_id in source_ids:
+ client_id = self._SALT[:2] + sha1(''.join([self._SALT, clip_id, self._TOKEN, server_id, client_location, source_id, self._SALT, self._CLIENT_NAME]).encode('utf-8')).hexdigest()
+ urls = self._download_json(
+ 'http://vas.sim-technik.de/vas/live/v2/videos/%s/sources/url' % clip_id,
+ clip_id, 'Downloading urls JSON', fatal=False, query={
+ 'access_token': self._TOKEN,
+ 'client_id': client_id,
+ 'client_location': client_location,
+ 'client_name': self._CLIENT_NAME,
+ 'server_id': server_id,
+ 'source_ids': source_id,
+ })
+ if not urls:
+ continue
+ if urls.get('status_code') != 0:
+ raise ExtractorError('This video is unavailable', expected=True)
+ urls_sources = urls['sources']
+ if isinstance(urls_sources, dict):
+ urls_sources = urls_sources.values()
+ for source in urls_sources:
+ source_url = source.get('url')
+ if not source_url:
+ continue
+ protocol = source.get('protocol')
+ mimetype = source.get('mimetype')
+ if mimetype == 'application/f4m+xml' or 'f4mgenerator' in source_url or determine_ext(source_url) == 'f4m':
+ formats.extend(self._extract_f4m_formats(
+ source_url, clip_id, f4m_id='hds', fatal=False))
+ elif mimetype == 'application/x-mpegURL':
+ formats.extend(self._extract_m3u8_formats(
+ source_url, clip_id, 'mp4', 'm3u8_native',
+ m3u8_id='hls', fatal=False))
+ else:
+ tbr = fix_bitrate(source['bitrate'])
+ if protocol in ('rtmp', 'rtmpe'):
+ mobj = re.search(r'^(?P<url>rtmpe?://[^/]+)/(?P<path>.+)$', source_url)
+ if not mobj:
+ continue
+ path = mobj.group('path')
+ mp4colon_index = path.rfind('mp4:')
+ app = path[:mp4colon_index]
+ play_path = path[mp4colon_index:]
+ formats.append({
+ 'url': '%s/%s' % (mobj.group('url'), app),
+ 'app': app,
+ 'play_path': play_path,
+ 'player_url': 'http://livepassdl.conviva.com/hf/ver/2.79.0.17083/LivePassModuleMain.swf',
+ 'page_url': 'http://www.prosieben.de',
+ 'tbr': tbr,
+ 'ext': 'flv',
+ 'format_id': 'rtmp%s' % ('-%d' % tbr if tbr else ''),
+ })
+ else:
+ formats.append({
+ 'url': source_url,
+ 'tbr': tbr,
+ 'format_id': 'http%s' % ('-%d' % tbr if tbr else ''),
+ })
+ self._sort_formats(formats)
+
+ return {
+ 'duration': duration,
+ 'formats': formats,
+ }
+
+
+class ProSiebenSat1IE(ProSiebenSat1BaseIE):
IE_NAME = 'prosiebensat1'
IE_DESC = 'ProSiebenSat.1 Digital'
_VALID_URL = r'https?://(?:www\.)?(?:(?:prosieben|prosiebenmaxx|sixx|sat1|kabeleins|the-voice-of-germany|7tv)\.(?:de|at|ch)|ran\.de|fem\.com)/(?P<id>.+)'
},
]
+ _TOKEN = 'prosieben'
+ _SALT = '01!8d8F_)r9]4s[qeuXfP%'
+ _CLIENT_NAME = 'kolibri-2.0.19-splec4'
_CLIPID_REGEXES = [
r'"clip_id"\s*:\s+"(\d+)"',
r'clipid: "(\d+)"',
def _extract_clip(self, url, webpage):
clip_id = self._html_search_regex(
self._CLIPID_REGEXES, webpage, 'clip id')
-
- access_token = 'prosieben'
- client_name = 'kolibri-2.0.19-splec4'
- client_location = url
-
- video = self._download_json(
- 'http://vas.sim-technik.de/vas/live/v2/videos',
- clip_id, 'Downloading videos JSON', query={
- 'access_token': access_token,
- 'client_location': client_location,
- 'client_name': client_name,
- 'ids': clip_id,
- })[0]
-
- if video.get('is_protected') is True:
- raise ExtractorError('This video is DRM protected.', expected=True)
-
- duration = float_or_none(video.get('duration'))
- source_ids = [compat_str(source['id']) for source in video['sources']]
-
- g = '01!8d8F_)r9]4s[qeuXfP%'
- client_id = g[:2] + sha1(''.join([clip_id, g, access_token, client_location, g, client_name]).encode('utf-8')).hexdigest()
-
- sources = self._download_json(
- 'http://vas.sim-technik.de/vas/live/v2/videos/%s/sources' % clip_id,
- clip_id, 'Downloading sources JSON', query={
- 'access_token': access_token,
- 'client_id': client_id,
- 'client_location': client_location,
- 'client_name': client_name,
- })
- server_id = sources['server_id']
-
title = self._html_search_regex(self._TITLE_REGEXES, webpage, 'title')
-
- def fix_bitrate(bitrate):
- bitrate = int_or_none(bitrate)
- if not bitrate:
- return None
- return (bitrate // 1000) if bitrate % 1000 == 0 else bitrate
-
- formats = []
- for source_id in source_ids:
- client_id = g[:2] + sha1(''.join([g, clip_id, access_token, server_id, client_location, source_id, g, client_name]).encode('utf-8')).hexdigest()
- urls = self._download_json(
- 'http://vas.sim-technik.de/vas/live/v2/videos/%s/sources/url' % clip_id,
- clip_id, 'Downloading urls JSON', fatal=False, query={
- 'access_token': access_token,
- 'client_id': client_id,
- 'client_location': client_location,
- 'client_name': client_name,
- 'server_id': server_id,
- 'source_ids': source_id,
- })
- if not urls:
- continue
- if urls.get('status_code') != 0:
- raise ExtractorError('This video is unavailable', expected=True)
- urls_sources = urls['sources']
- if isinstance(urls_sources, dict):
- urls_sources = urls_sources.values()
- for source in urls_sources:
- source_url = source.get('url')
- if not source_url:
- continue
- protocol = source.get('protocol')
- mimetype = source.get('mimetype')
- if mimetype == 'application/f4m+xml' or 'f4mgenerator' in source_url or determine_ext(source_url) == 'f4m':
- formats.extend(self._extract_f4m_formats(
- source_url, clip_id, f4m_id='hds', fatal=False))
- elif mimetype == 'application/x-mpegURL':
- formats.extend(self._extract_m3u8_formats(
- source_url, clip_id, 'mp4', 'm3u8_native',
- m3u8_id='hls', fatal=False))
- else:
- tbr = fix_bitrate(source['bitrate'])
- if protocol in ('rtmp', 'rtmpe'):
- mobj = re.search(r'^(?P<url>rtmpe?://[^/]+)/(?P<path>.+)$', source_url)
- if not mobj:
- continue
- path = mobj.group('path')
- mp4colon_index = path.rfind('mp4:')
- app = path[:mp4colon_index]
- play_path = path[mp4colon_index:]
- formats.append({
- 'url': '%s/%s' % (mobj.group('url'), app),
- 'app': app,
- 'play_path': play_path,
- 'player_url': 'http://livepassdl.conviva.com/hf/ver/2.79.0.17083/LivePassModuleMain.swf',
- 'page_url': 'http://www.prosieben.de',
- 'tbr': tbr,
- 'ext': 'flv',
- 'format_id': 'rtmp%s' % ('-%d' % tbr if tbr else ''),
- })
- else:
- formats.append({
- 'url': source_url,
- 'tbr': tbr,
- 'format_id': 'http%s' % ('-%d' % tbr if tbr else ''),
- })
- self._sort_formats(formats)
-
+ info = self._extract_video_info(url, clip_id)
description = self._html_search_regex(
self._DESCRIPTION_REGEXES, webpage, 'description', fatal=False)
thumbnail = self._og_search_thumbnail(webpage)
upload_date = unified_strdate(self._html_search_regex(
self._UPLOAD_DATE_REGEXES, webpage, 'upload date', default=None))
- return {
+ info.update({
'id': clip_id,
'title': title,
'description': description,
'thumbnail': thumbnail,
'upload_date': upload_date,
- 'duration': duration,
- 'formats': formats,
- }
+ })
+ return info
def _extract_playlist(self, url, webpage):
playlist_id = self._html_search_regex(
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
-from .common import InfoExtractor
+from .prosiebensat1 import ProSiebenSat1BaseIE
from ..utils import (
- ExtractorError,
unified_strdate,
- int_or_none,
+ parse_duration,
+ compat_str,
)
-class Puls4IE(InfoExtractor):
- _VALID_URL = r'https?://(?:www\.)?puls4\.com/video/[^/]+/play/(?P<id>[0-9]+)'
+class Puls4IE(ProSiebenSat1BaseIE):
+ _VALID_URL = r'https?://(?:www\.)?puls4\.com/(?P<id>(?:[^/]+/)*?videos/[^?#]+)'
_TESTS = [{
- 'url': 'http://www.puls4.com/video/pro-und-contra/play/2716816',
- 'md5': '49f6a6629747eeec43cef6a46b5df81d',
+ 'url': 'http://www.puls4.com/2-minuten-2-millionen/staffel-3/videos/2min2miotalk/Tobias-Homberger-von-myclubs-im-2min2miotalk-118118',
+ 'md5': 'fd3c6b0903ac72c9d004f04bc6bb3e03',
'info_dict': {
- 'id': '2716816',
- 'ext': 'mp4',
- 'title': 'Pro und Contra vom 23.02.2015',
- 'description': 'md5:293e44634d9477a67122489994675db6',
- 'duration': 2989,
- 'upload_date': '20150224',
+ 'id': '118118',
+ 'ext': 'flv',
+ 'title': 'Tobias Homberger von myclubs im #2min2miotalk',
+ 'description': 'md5:f9def7c5e8745d6026d8885487d91955',
+ 'upload_date': '20160830',
'uploader': 'PULS_4',
},
- 'skip': 'Only works from Germany',
- }, {
- 'url': 'http://www.puls4.com/video/kult-spielfilme/play/1298106',
- 'md5': '6a48316c8903ece8dab9b9a7bf7a59ec',
- 'info_dict': {
- 'id': '1298106',
- 'ext': 'mp4',
- 'title': 'Lucky Fritz',
- },
- 'skip': 'Only works from Germany',
}]
+ _TOKEN = 'puls4'
+ _SALT = '01!kaNgaiNgah1Ie4AeSha'
+ _CLIENT_NAME = ''
def _real_extract(self, url):
- video_id = self._match_id(url)
- webpage = self._download_webpage(url, video_id)
-
- error_message = self._html_search_regex(
- r'<div[^>]+class="message-error"[^>]*>(.+?)</div>',
- webpage, 'error message', default=None)
- if error_message:
- raise ExtractorError(
- '%s returned error: %s' % (self.IE_NAME, error_message), expected=True)
-
- real_url = self._html_search_regex(
- r'\"fsk-button\".+?href=\"([^"]+)',
- webpage, 'fsk_button', default=None)
- if real_url:
- webpage = self._download_webpage(real_url, video_id)
-
- player = self._search_regex(
- r'p4_video_player(?:_iframe)?\("video_\d+_container"\s*,(.+?)\);\s*\}',
- webpage, 'player')
-
- player_json = self._parse_json(
- '[%s]' % player, video_id,
- transform_source=lambda s: s.replace('undefined,', ''))
-
- formats = None
- result = None
-
- for v in player_json:
- if isinstance(v, list) and not formats:
- formats = [{
- 'url': f['url'],
- 'format': 'hd' if f.get('hd') else 'sd',
- 'width': int_or_none(f.get('size_x')),
- 'height': int_or_none(f.get('size_y')),
- 'tbr': int_or_none(f.get('bitrate')),
- } for f in v]
- self._sort_formats(formats)
- elif isinstance(v, dict) and not result:
- result = {
- 'id': video_id,
- 'title': v['videopartname'].strip(),
- 'description': v.get('videotitle'),
- 'duration': int_or_none(v.get('videoduration') or v.get('episodeduration')),
- 'upload_date': unified_strdate(v.get('clipreleasetime')),
- 'uploader': v.get('channel'),
- }
-
- result['formats'] = formats
-
- return result
+ path = self._match_id(url)
+ content_path = self._download_json(
+ 'http://www.puls4.com/api/json-fe/page/' + path, path)['content'][0]['url']
+ media = self._download_json(
+ 'http://www.puls4.com' + content_path,
+ content_path)['mediaCurrent']
+ player_content = media['playerContent']
+ info = self._extract_video_info(url, player_content['id'])
+ info.update({
+ 'id': compat_str(media['objectId']),
+ 'title': player_content['title'],
+ 'description': media.get('description'),
+ 'thumbnail': media.get('previewLink'),
+ 'upload_date': unified_strdate(media.get('date')),
+ 'duration': parse_duration(player_content.get('duration')),
+ 'episode': player_content.get('episodePartName'),
+ 'show': media.get('channel'),
+ 'season_id': player_content.get('seasonId'),
+ 'uploader': player_content.get('sourceCompany'),
+ })
+ return info
--- /dev/null
+# encoding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from .brightcove import BrightcoveLegacyIE
+from ..compat import (
+ compat_parse_qs,
+ compat_urlparse,
+)
+
+
+class RMCDecouverteIE(InfoExtractor):
+ _VALID_URL = r'https?://rmcdecouverte\.bfmtv\.com/mediaplayer-replay.*?\bid=(?P<id>\d+)'
+
+ _TEST = {
+ 'url': 'http://rmcdecouverte.bfmtv.com/mediaplayer-replay/?id=1430&title=LES%20HEROS%20DU%2088e%20ETAGE',
+ 'info_dict': {
+ 'id': '5111223049001',
+ 'ext': 'mp4',
+ 'title': ': LES HEROS DU 88e ETAGE',
+ 'description': 'Découvrez comment la bravoure de deux hommes dans la Tour Nord du World Trade Center a sauvé la vie d\'innombrables personnes le 11 septembre 2001.',
+ 'uploader_id': '1969646226001',
+ 'upload_date': '20160904',
+ 'timestamp': 1472951103,
+ },
+ 'params': {
+ # rtmp download
+ 'skip_download': True,
+ },
+ 'skip': 'Only works from France',
+ }
+ BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/1969646226001/default_default/index.html?videoId=%s'
+
+ def _real_extract(self, url):
+ video_id = self._match_id(url)
+ webpage = self._download_webpage(url, video_id)
+ brightcove_legacy_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
+ brightcove_id = compat_parse_qs(compat_urlparse.urlparse(brightcove_legacy_url).query)['@videoPlayer'][0]
+ return self.url_result(self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id, 'BrightcoveNew', brightcove_id)
class RutubeEmbedIE(InfoExtractor):
IE_NAME = 'rutube:embed'
IE_DESC = 'Rutube embedded videos'
- _VALID_URL = 'https?://rutube\.ru/(?:video|play)/embed/(?P<id>[0-9]+)'
+ _VALID_URL = r'https?://rutube\.ru/(?:video|play)/embed/(?P<id>[0-9]+)'
_TESTS = [{
'url': 'http://rutube.ru/video/embed/6722881?vk_puid37=&vk_puid38=',
class SpiegelArticleIE(InfoExtractor):
- _VALID_URL = 'https?://www\.spiegel\.de/(?!video/)[^?#]*?-(?P<id>[0-9]+)\.html'
+ _VALID_URL = r'https?://www\.spiegel\.de/(?!video/)[^?#]*?-(?P<id>[0-9]+)\.html'
IE_NAME = 'Spiegel:Article'
IE_DESC = 'Articles on spiegel.de'
_TESTS = [{
--- /dev/null
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import int_or_none
+
+
+class TeleQuebecIE(InfoExtractor):
+ _VALID_URL = r'https?://zonevideo\.telequebec\.tv/media/(?P<id>\d+)'
+ _TEST = {
+ 'url': 'http://zonevideo.telequebec.tv/media/20984/le-couronnement-de-new-york/couronnement-de-new-york',
+ 'md5': 'fe95a0957e5707b1b01f5013e725c90f',
+ 'info_dict': {
+ 'id': '20984',
+ 'ext': 'mp4',
+ 'title': 'Le couronnement de New York',
+ 'description': 'md5:f5b3d27a689ec6c1486132b2d687d432',
+ 'upload_date': '20160220',
+ 'timestamp': 1455965438,
+ }
+ }
+
+ def _real_extract(self, url):
+ media_id = self._match_id(url)
+ media_data = self._download_json(
+ 'https://mnmedias.api.telequebec.tv/api/v2/media/' + media_id,
+ media_id)['media']
+ return {
+ '_type': 'url_transparent',
+ 'id': media_id,
+ 'url': 'limelight:media:' + media_data['streamInfo']['sourceId'],
+ 'title': media_data['title'],
+ 'description': media_data.get('descriptions', [{'text': None}])[0].get('text'),
+ 'duration': int_or_none(media_data.get('durationInMilliseconds'), 1000),
+ 'ie_key': 'LimelightMedia',
+ }
# encoding: utf-8
from __future__ import unicode_literals
+
import re
from .common import InfoExtractor
from .brightcove import BrightcoveLegacyIE
-from ..compat import compat_parse_qs
+from ..compat import (
+ compat_parse_qs,
+ compat_urlparse,
+)
class TlcDeIE(InfoExtractor):
title = mobj.group('title')
webpage = self._download_webpage(url, title)
brightcove_legacy_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
- brightcove_id = compat_parse_qs(brightcove_legacy_url)['@videoPlayer'][0]
+ brightcove_id = compat_parse_qs(compat_urlparse.urlparse(brightcove_legacy_url).query)['@videoPlayer'][0]
return self.url_result(self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id, 'BrightcoveNew', brightcove_id)
class TVNoeIE(JWPlatformBaseIE):
- _VALID_URL = r'https?://(www\.)?tvnoe\.cz/video/(?P<id>[0-9]+)'
+ _VALID_URL = r'https?://(?:www\.)?tvnoe\.cz/video/(?P<id>[0-9]+)'
_TEST = {
'url': 'http://www.tvnoe.cz/video/10362',
'md5': 'aee983f279aab96ec45ab6e2abb3c2ca',
if thumbnail:
video_id = self._search_regex(
r'https?://[^/]+/imagecache/(?:[^/]+/)+seasons/\d+/(\d{6,})/',
- thumbnail, 'video id', default=None)
+ thumbnail, 'video id', default=None)
if not video_id:
video_id = self._search_regex(
class TwitterAmplifyIE(TwitterBaseIE):
IE_NAME = 'twitter:amplify'
- _VALID_URL = 'https?://amp\.twimg\.com/v/(?P<id>[0-9a-f\-]{36})'
+ _VALID_URL = r'https?://amp\.twimg\.com/v/(?P<id>[0-9a-f\-]{36})'
_TEST = {
'url': 'https://amp.twimg.com/v/0ba0c3c7-0af3-4c0a-bed5-7efd1ffa2951',
from .common import InfoExtractor
from ..utils import (
int_or_none,
- parse_age_limit,
- parse_iso8601,
+ xpath_element,
xpath_text,
)
_VALID_URL = r'videomore:(?P<sid>\d+)$|https?://videomore\.ru/(?:(?:embed|[^/]+/[^/]+)/|[^/]+\?.*\btrack_id=)(?P<id>\d+)(?:[/?#&]|\.(?:xml|json)|$)'
_TESTS = [{
'url': 'http://videomore.ru/kino_v_detalayah/5_sezon/367617',
- 'md5': '70875fbf57a1cd004709920381587185',
+ 'md5': '44455a346edc0d509ac5b5a5b531dc35',
'info_dict': {
'id': '367617',
'ext': 'flv',
- 'title': 'В гостях Алексей Чумаков и Юлия Ковальчук',
- 'description': 'В гостях – лучшие романтические комедии года, «Выживший» Иньярриту и «Стив Джобс» Дэнни Бойла.',
+ 'title': 'Кино в деталях 5 сезон В гостях Алексей Чумаков и Юлия Ковальчук',
'series': 'Кино в деталях',
'episode': 'В гостях Алексей Чумаков и Юлия Ковальчук',
- 'episode_number': None,
- 'season': 'Сезон 2015',
- 'season_number': 5,
'thumbnail': 're:^https?://.*\.jpg',
'duration': 2910,
- 'age_limit': 16,
'view_count': int,
+ 'comment_count': int,
+ 'age_limit': 16,
},
}, {
'url': 'http://videomore.ru/embed/259974',
'info_dict': {
'id': '259974',
'ext': 'flv',
- 'title': '80 серия',
- 'description': '«Медведей» ждет решающий матч. Макеев выясняет отношения со Стрельцовым. Парни узнают подробности прошлого Макеева.',
+ 'title': 'Молодежка 2 сезон 40 серия',
'series': 'Молодежка',
- 'episode': '80 серия',
- 'episode_number': 40,
- 'season': '2 сезон',
- 'season_number': 2,
+ 'episode': '40 серия',
'thumbnail': 're:^https?://.*\.jpg',
'duration': 2809,
- 'age_limit': 16,
'view_count': int,
+ 'comment_count': int,
+ 'age_limit': 16,
},
'params': {
'skip_download': True,
'info_dict': {
'id': '341073',
'ext': 'flv',
- 'title': 'Команда проиграла из-за Бакина?',
- 'description': 'Молодежка 3 сезон скоро',
- 'series': 'Молодежка',
+ 'title': 'Промо Команда проиграла из-за Бакина?',
'episode': 'Команда проиграла из-за Бакина?',
- 'episode_number': None,
- 'season': 'Промо',
- 'season_number': 99,
'thumbnail': 're:^https?://.*\.jpg',
'duration': 29,
'age_limit': 16,
'http://videomore.ru/video/tracks/%s.xml' % video_id,
video_id, 'Downloading video XML')
- video_url = xpath_text(video, './/video_url', 'video url', fatal=True)
+ item = xpath_element(video, './/playlist/item', fatal=True)
+
+ title = xpath_text(
+ item, ('./title', './episode_name'), 'title', fatal=True)
+
+ video_url = xpath_text(item, './video_url', 'video url', fatal=True)
formats = self._extract_f4m_formats(video_url, video_id, f4m_id='hds')
self._sort_formats(formats)
- data = self._download_json(
- 'http://videomore.ru/video/tracks/%s.json' % video_id,
- video_id, 'Downloading video JSON')
-
- title = data.get('title') or data['project_title']
- description = data.get('description') or data.get('description_raw')
- timestamp = parse_iso8601(data.get('published_at'))
- duration = int_or_none(data.get('duration'))
- view_count = int_or_none(data.get('views'))
- age_limit = parse_age_limit(data.get('min_age'))
- thumbnails = [{
- 'url': thumbnail,
- } for thumbnail in data.get('big_thumbnail_urls', [])]
-
- series = data.get('project_title')
- episode = data.get('title')
- episode_number = int_or_none(data.get('episode_of_season') or None)
- season = data.get('season_title')
- season_number = int_or_none(data.get('season_pos') or None)
+ thumbnail = xpath_text(item, './thumbnail_url')
+ duration = int_or_none(xpath_text(item, './duration'))
+ view_count = int_or_none(xpath_text(item, './views'))
+ comment_count = int_or_none(xpath_text(item, './count_comments'))
+ age_limit = int_or_none(xpath_text(item, './min_age'))
+
+ series = xpath_text(item, './project_name')
+ episode = xpath_text(item, './episode_name')
return {
'id': video_id,
'title': title,
- 'description': description,
'series': series,
'episode': episode,
- 'episode_number': episode_number,
- 'season': season,
- 'season_number': season_number,
- 'thumbnails': thumbnails,
- 'timestamp': timestamp,
+ 'thumbnail': thumbnail,
'duration': duration,
'view_count': view_count,
+ 'comment_count': comment_count,
'age_limit': age_limit,
'formats': formats,
}
def extract_url(path_template, url_type):
req_url = 'http://www.wat.tv/get/%s' % (path_template % video_id)
- head = self._request_webpage(HEADRequest(req_url), video_id, 'Extracting %s url' % url_type)
- red_url = head.geturl()
- if req_url == red_url:
- raise ExtractorError(
- '%s said: Sorry, this video is not available from your country.' % self.IE_NAME,
- expected=True)
- return red_url
+ head = self._request_webpage(HEADRequest(req_url), video_id, 'Extracting %s url' % url_type, fatal=False)
+ if head:
+ red_url = head.geturl()
+ if req_url != red_url:
+ return red_url
+ return None
+
+ def remove_bitrate_limit(manifest_url):
+ return re.sub(r'(?:max|min)_bitrate=\d+&?', '', manifest_url)
formats = []
try:
- http_url = extract_url('android5/%s.mp4', 'http')
- m3u8_url = extract_url('ipad/%s.m3u8', 'm3u8')
- m3u8_formats = self._extract_m3u8_formats(
- m3u8_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls')
- formats.extend(m3u8_formats)
- formats.extend(self._extract_f4m_formats(
- m3u8_url.replace('ios.', 'web.').replace('.m3u8', '.f4m'),
- video_id, f4m_id='hds', fatal=False))
- for m3u8_format in m3u8_formats:
- vbr, abr = m3u8_format.get('vbr'), m3u8_format.get('abr')
- if not vbr or not abr:
- continue
- format_id = m3u8_format['format_id'].replace('hls', 'http')
- fmt_url = re.sub(r'%s-\d+00-\d+' % video_id, '%s-%d00-%d' % (video_id, round(vbr / 100), round(abr)), http_url)
- if self._is_valid_url(fmt_url, video_id, format_id):
- f = m3u8_format.copy()
- f.update({
- 'url': fmt_url,
- 'format_id': format_id,
- 'protocol': 'http',
- })
- formats.append(f)
+ manifest_urls = self._download_json(
+ 'http://www.wat.tv/get/webhtml/' + video_id, video_id)
+ m3u8_url = manifest_urls.get('hls')
+ if m3u8_url:
+ m3u8_url = remove_bitrate_limit(m3u8_url)
+ m3u8_formats = self._extract_m3u8_formats(
+ m3u8_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls', fatal=False)
+ if m3u8_formats:
+ formats.extend(m3u8_formats)
+ formats.extend(self._extract_f4m_formats(
+ m3u8_url.replace('ios', 'web').replace('.m3u8', '.f4m'),
+ video_id, f4m_id='hds', fatal=False))
+ http_url = extract_url('android5/%s.mp4', 'http')
+ if http_url:
+ for m3u8_format in m3u8_formats:
+ vbr, abr = m3u8_format.get('vbr'), m3u8_format.get('abr')
+ if not vbr or not abr:
+ continue
+ format_id = m3u8_format['format_id'].replace('hls', 'http')
+ fmt_url = re.sub(r'%s-\d+00-\d+' % video_id, '%s-%d00-%d' % (video_id, round(vbr / 100), round(abr)), http_url)
+ if self._is_valid_url(fmt_url, video_id, format_id):
+ f = m3u8_format.copy()
+ f.update({
+ 'url': fmt_url,
+ 'format_id': format_id,
+ 'protocol': 'http',
+ })
+ formats.append(f)
+ mpd_url = manifest_urls.get('mpd')
+ if mpd_url:
+ formats.extend(self._extract_mpd_formats(remove_bitrate_limit(
+ mpd_url), video_id, mpd_id='dash', fatal=False))
self._sort_formats(formats)
except ExtractorError:
abr = 64
determine_ext,
)
-from .brightcove import BrightcoveNewIE
+from .brightcove import (
+ BrightcoveLegacyIE,
+ BrightcoveNewIE,
+)
from .nbc import NBCSportsVPlayerIE
if nbc_sports_url:
return self.url_result(nbc_sports_url, NBCSportsVPlayerIE.ie_key())
+ # Look for Brightcove Legacy Studio embeds
+ bc_url = BrightcoveLegacyIE._extract_brightcove_url(webpage)
+ if bc_url:
+ return self.url_result(bc_url, BrightcoveLegacyIE.ie_key())
+
# Look for Brightcove New Studio embeds
bc_url = BrightcoveNewIE._extract_url(webpage)
if bc_url:
class YoutubeHistoryIE(YoutubeFeedsInfoExtractor):
IE_DESC = 'Youtube watch history, ":ythistory" for short (requires authentication)'
- _VALID_URL = 'https?://www\.youtube\.com/feed/history|:ythistory'
+ _VALID_URL = r'https?://www\.youtube\.com/feed/history|:ythistory'
_FEED_NAME = 'history'
_PLAYLIST_TITLE = 'Youtube History'
from __future__ import unicode_literals
-__version__ = '2016.09.04.1'
+__version__ = '2016.09.08'
projects / youtube-dl / commitdiff