Mythtv using Schedules Direct in the UK

Schedules Direct

Now that Radio times listings are now longer updated I needed to move to Mythtv using Schedules Direct in the UK. This is mostly a direct lift from the post https://www.mythtv.org/wiki/Migrating_to_Schedules_Direct_in_the_UK Go there and follow it. This post is for me in case it disappears.

You will need an account at schedules direct. They offer a 7 day trial, use that while you are getting it going. the get the one year membership it costs $25 which is a bout £18-£19. Remember the user name and password you need it below and next year when you want to renew. There is nothing else you need here. The conversion for MythTV is all done by the grabber script.

TV grabber JSON

To install tv_grab_zz_json with Ubuntu you need so software before it will work.

sudo apt-get install xmltv
sudo apt-get install libdatetime-format-dateparse-perl

cpan
> install XML::TreePP
> install Lingua::EN::Numbers::Ordinate
> exit

Now download the software from http://snapshot.xmltv.org/ gives you a zip file. Unzip it somewhere handy so you can see the contents. The README file is pretty good read it.

In short you need to run the following. I entered the schedules direct grabber, tv_grab_zz_sdjson, at the first prompt.

perl ./Makefile.PL
make
make test
sudo make install

This will install a bunch of files in /usr/local/bin, we will need tv_grab_zz_sdjson

Configure xmltv

We need to configure the feed so it pulls the data through for our region and gives the channels we want. It is important you get this bit right or your channel data will never get filled. You may even missout some channels and never get data for them.

tv_grab_zz_sdjson --configure

When this runs you will be asked a few questions.

  • Cache File: Press Return. This chooses the defaults which work. I change the directory part to point at HOME/.mythtv This keeps all the mythtv files together. Your choice.
  • Channel ID format: Select 2. MythTV Internal DD Grabber Format (eg: 12345)
  • Select previously shown format: Select 0. Date Only
  • Username: Your Schedules Direct user name.
  • Password: Your Schedules Direct password.

Some files get downloaded at this point. They should all be “200 OK”. Then some instructions about adding up to 4 lineups.

** POST https://json.schedulesdirect.org/20141201/token ==> 200 OK (1s)
** GET https://json.schedulesdirect.org/20141201/status ==> 200 OK
** GET https://json.schedulesdirect.org/20141201/lineups ==> 200 OK (1s)
This step configures the lineups enabled for your Schedules Direct account. It impacts all other configurations and programs using the JSON API with your account. A maximum of 4 lineups can by added to your account. In a later step you will choose which lineups or channels to actually use for this configuration.
Current lineups enabled for your Schedules Direct account:
#. Lineup ID | Name | Location | Transport
  • Edit account lineups: Press Return. This will allow you to add some lineups
  • Lineup ID or Country (ISO-3166-1 alpha 3 such as USA or CAN): GBR This is for Great Britain.

Some more files get downloaded all of them should have a 200 OK code.

** GET https://json.schedulesdirect.org/20141201/available ==> 200 OK
  • Search by Transmitter or Postal Code: [transmitter,postal (default=transmitter)]: postal. I find it simpler to use my post cod here. Let the scriopt work out the best area.
  • Postal Code: XX00 Where XX00 is the first part of your UK post code.

Another file gets downloaded for a list of suppliers in your area. Again the status should be 200 OK.

** GET https://json.schedulesdirect.org/20141201/headends?country=GBR&postalcode=XX00 ==> 200 OK (1s)
#. Lineup ID | Name | Location | Transport
1. GBR-1000013-DEFAULT | BT TV - Cable | London | Cable
2. GBR-1000014-DEFAULT | Freeview - Cable | London | Cable
3. GBR-1000015-DEFAULT | Virgin - Cable | London | Cable
4. GBR-1000016-DEFAULT | Sky SD - Cable | London | Cable
5. GBR-1000017-DEFAULT | Freesat - Cable | London | Cable
6. GBR-1000118-DEFAULT | United Kingdom - Cable | National | Cable
7. GBR-1000193-DEFAULT | United Kingdom (extended) - Cable | National | Cable
8. GBR-1000197-DEFAULT | Sky HD - Cable | London | Cable
9. GBR-1000449-DEFAULT | Sky SD - Cable | National | Cable
10. GBR-1000450-DEFAULT | Sky HD - Cable | National | Cable
11. GBR-1002100-DEFAULT | Freesat from Sky - Cable | National | Cable
12. GBR-1002101-DEFAULT | Talk Talk - Cable | National | Cable
13. GBR-1002757-DEFAULT | YouView - Cable | London | Cable
  • Add lineup (0 = none): [0,1,2,3,4,5,6,7,8,9,10,11,12,13 (default=0)]:5 I’m configuring for FreeSat. The FreeSat from those nasty people at Sky has missing channels.
** PUT https://json.schedulesdirect.org/20141201/lineups/GBR-1000017-DEFAULT ==> 200 OK (1s)
** GET https://json.schedulesdirect.org/20141201/lineups ==> 200 OK
This step configures the lineups enabled for your Schedules Direct account. It impacts all other configurations and programs using the JSON API with your account. A maximum of 4 lineups can by added to your account. In a later step you will choose which lineups or channels to actually use for this configuration.
Current lineups enabled for your Schedules Direct account:
#. Lineup ID | Name | Location | Transport
1. GBR-1000017-DEFAULT | Freesat - Cable | London | Cable
  • Edit account lineups: [continue,add,delete (default=continue)] Press Return This is all I’m configuring here. FreeView has its own configuration, in which case choose 2.
  • Select mode: [lineups,channels (default=lineups)]: Press Return This will select to download all channel data. I find it simpler than editing the list of channels by hand.
** GET https://json.schedulesdirect.org/20141201/lineups ==> 200 OK (1s)
Choose lineups to use for this configuration.
  • GBR-1000017-DEFAULT [yes,no,all,none (default=no)]: all. I want the entire listing

The generated File

This should give you something like this:

cat ~/.xmltv/tv_grab_zz_sdjson.conf 
cache=/home/LINUXNAME/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=mythtv
previously-shown-format=date
username=USERNAME
password=PASSWORD
mode=lineup
lineup=GBR-1000017-DEFAULT

Your values for LINUXNAME, USERNAME, PASSWORD and GBR-1000017-DEFAULT will need to be adjusted to suit your inputs.

We can either copy the new tv_grab_zz_sdjson.conf config file or create a sym link either works.

cp ~/.xmltv/tv_grab_zz_sdjson.conf ~/.mythtv/SOURCENAME.xmltv

I rename the file ~/.mythtv/SOURCENAME.xmltv to FreeSat.xmltv as that is the name of the source and it is what Mythtv expects to find when you run mythfilldatabase.

I can also get terrestrial TV, So I run the the –configure twice once for FreeSat as above and again for FreeView. I run it for Freeview cable and also a third time for using transmitter not postcode. I then select the Guildford transmitter. I found i had missing channels on FreeView. I had ITV3 +1 but not ITV3 go figure.

Remember to copy the tv_grab_zz_sdjson.conf file after each the same file is used for both. The only difference are the values for cache and lineup.

Test this new config file

You can now test the configuration you just set up is at least looking good by doing a test download.

tv_grab_zz_sdjson --config-file ~/.xmltv/tv_grab_zz_sdjson.conf --output /tmp/xmltv_grab.xml

The output should be similar to the following. It takes a few minutes to download and process the data which is an xml file.

Initializing...
** POST https://json.schedulesdirect.org/20141201/token ==> 200 OK
** GET https://json.schedulesdirect.org/20141201/status ==> 200 OK (1s)
Updating lineups...
Indexing channels...
Updating schedules...
** POST https://json.schedulesdirect.org/20141201/schedules/md5 ==> 200 OK (2s)
Updating programs...
Removing old cache entries...
Writing output...
Done

The resulting text file should look like the following snippet. The top part has data for the channels the remaining is program listings.

<!DOCTYPE tv SYSTEM "xmltv.dtd">

<tv source-info-url="http://www.schedulesdirect.org" source-info-name="Schedules Direct" generator-info-name="tv_grab_zz_sdjson 1.1 2017/01/24 00:41:08">
  <channel id="101972">
    <display-name>BLAZE
    <display-name>BLAZUK
    <display-name>063
    <icon src="https://s3.amazonaws.com/schedulesdirect/assets/stationLogos/s101972_h3_aa.png" width="360" height="270" />
  </channel>
  <channel id="102340">
    <display-name>Sewing Quarter
    <display-name>SEWQUA
    <display-name>078
    <icon height="270" width="360" src="https://s3.amazonaws.com/schedulesdirect/assets/stationLogos/s102340_h3_aa.png" />
  </channel>

Remember to delete /tmp/xmltv_grab.xml when you have looked at it. We do not need it any more.

Backup Mythconverg database

The simplest way is to use the mysqldump, or you can use the built in scripts for Mythtv.

Running mythfilldatabase

before running mythfilldatabase you will needto tell Mythtv which grabber to use. The handy command tv_find_grabbers will list all the known grabbers and what they are called in Mythtv. Remember above we used tv_grab_zz_sdjson to generate the FreeSat.xmltv config file. So look for that in the output.

/usr/bin/tv_grab_zz_sdjson|Schedules Direct JSON API

run Mythtvbackend and go to 4. Video Sources, Then update the video sources to point at the right grabber. On the next screen changethe listings grabber to be Schedules Direct JSON API.

Now you can run mythfilldatabase to update the channel listings. This takes a few minutes to complete.

NOTE: The original post talked about removing old program IDs, I found this made no difference either way. After running mythfilldatabase check your up coming recordings in Mythtv there will be some that have previously been recorded.

So there you have it how to setup Mythtv using Schedules Direct in the UK.

One thought on “Mythtv using Schedules Direct in the UK

  1. tom

    Great post, the other posts I found seem to skip over important parts leaving you to work out what to do. You take us through step by step. Thanks again.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *