How to extract m3u8 url from streaming video site?

[Bo Berglund]

On Fri, 8 Apr 2022 10:50:57 -0700, MR ZenWiz <mrzenwiz at gmail.com> wrote:

>On Fri, Apr 8, 2022 at 2:12 AM Bo Berglund <bo.berglund at gmail.com> wrote:
>>
>> I would like to automate the extraction of the m3u8 URL to a video player on a
>> webpage.
>>
>I have found YouTube-dl to be an excellent downloader for most sites,
>unless they have serious security. You can even pass in a userid and
>password for videos that require them.
>

This site does not use a login so that is not needed, however it plays the show
in real time inside a video player on the page. And this makse it 
I have used youtube-dl in the past but it is no longer supported (abandoned
since more than a year, I believe).
But I have tested it nevertheless:

$ youtube-dl -F http://www.freeintertv.com/view/id-2308
[generic] id-2308: Requesting header
WARNING: Falling back on generic information extractor.
[generic] id-2308: Downloading webpage
[generic] id-2308: Extracting information
ERROR: Unsupported URL: http://www.freeintertv.com/view/id-2308

The replacement yt-dlp works better on most places and is actively supported,
but still on this site it yields this:

$ yt-dlp -F http://www.freeintertv.com/view/id-2308
[generic] id-2308: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] id-2308: Downloading webpage
[generic] id-2308: Extracting information
ERROR: [generic] Unsupported URL: http://www.freeintertv.com/view/id-2308

But yt-dlp cannot handle real time streaming videos anyway, so I have had to
revert to using ffmpeg directly instead.
And ffmpeg works fine as long as one has access to the m3u8 URL that is behind
the video player on the webpage.
By using Firefox in debug mode (start playing the video on the page and hit F12,
then look in the Network tab and you will see a lot of activity and there is
also the m3u8 URL. Right click it and select Copy/Copy URL and you get the
source that will work with ffmpeg.

The problem in this case is that the site changes the URL at an interval of some
hours or so...
That is why I need a way to script the action I can do manually inside Firefox
so it can be automated.

Here are some of the different URL:s extracted during one day, look at the way
the very long identifiers change following =exp=:

"https://nbcnews3.akamaized.net/hls/live/723429/NBCNewsNowa418c506/clear/hdntl=exp=1649366407~acl=%2f*~data=hdntl~hmac=a1ca56c1de48b3817bd45da948c857c2c27a3db807cb>
"https://nbcnews3.akamaized.net/hls/live/723429/NBCNewsNowa418c506/clear/hdntl=exp=1649453786~acl=%2f*~data=hdntl~hmac=619b13e396e0cc96f3ea80d68d340a98d8db857d9764>
"https://nbcnews3.akamaized.net/hls/live/723429/NBCNewsNowa418c506/clear/hdntl=exp=1649494253~acl=%2f*~data=hdntl~hmac=5a922ab7e0efcaa1fd45f6f6ece1a51f360ffeba83bc>

Apart from downloading the video for later viewing the URL is also used on my
Kodi media player box for .strm files allowing realtime viewing of the show on
my TV, but the same here:
Once the site changes the URL I have to manually extract it and put it back into
my script that generates the .strm files for Kodi.


-- 
Bo Berglund
Developer in Sweden