XTVD-Tools Project

XTVD-Tools is a Python package containing APIs and associated command-line tools for searching and displaying information from Schedules Direct Extensible TV Data (XTVD) files containing TV listing information.

The latest version of this README file should be available here.

This package is released under the following non-commercial use license:

Copyright (c) 2005-2022 by Ron Frederick <ronf@timeheart.net>.
All rights reserved.

License is granted to copy, to use, and to make and use derivative works
for non-commercial purposes, provided that this copyright notice and
contact information is left intact in all such works and any altered
versions of the source are clearly marked as such.

This software is provided "AS IS", without warranty of any kind, express
or implied. In no event shall the author be held liable for any damages
in connection with this software or the use or other dealings in this
software.

Overview

Tribune Media provides a web service through which approximately 13 days of North American TV listing data can be downloaded in XML format. This package provides a tool to perform this download and import the data into a MySQL database. Once in database form, another tool is provided to search the listings and print out the associated programs and schedule information. For example:

% xtvd_query title 'Star Trek: Enterprise'

Star Trek: Enterprise: Divergence

   Archer pursues the kidnapped Dr. Phlox deep into Klingon
   territory; Dr. Phlox decides to cooperate with the Klingons and
   help find a cure to a virus that is ravaging one of their
   outposts.

   Actors: Anthony Montgomery, Connor Trinneer, Dominic Keating, John
           Billingsley, Jolene Blalock, Linda Park, Scott Bakula

   Director: Michael Grossman

   Executive Producers: Brannon Braga, Rick Berman

   Guest Stars: Ada Maris, Eric Pierpoint, James Avery, John Schuck,
                Kristin Bauer, Matt Jenkins, Terrell Tilford, Wayne
                Grace

   Writer: Mike Sussman

   Original Air Date: Fri Feb 25 2005

   Genres: Adventure, Fantasy, Science fiction

   Airings:
      Fri Feb 25  5:00 PM -  6:00 PM on 8105 WWOR    (CC, TV-PG)
      Fri Feb 25  5:00 PM -  6:00 PM on 8775 WSBK    (CC, TV-PG)
      Fri Feb 25  8:00 PM -  9:00 PM on 8225 KBHK    (CC, TV-PG)
      Sat Feb 26  7:00 PM -  8:00 PM on 8775 WSBK    (R, CC, TV-PG)
      Sun Feb 27  4:00 PM -  5:00 PM on 8105 WWOR    (CC, TV-PG)
      Sun Feb 27  8:00 PM -  9:00 PM on 8225 KBHK    (CC, TV-PG)

Star Trek: Enterprise: Hatchery

   The crew begins to question Archer's judgment when he goes to
   extremes to save an abandoned nest of Xindi-Insectoid eggs.

   Actors: Anthony Montgomery, Connor Trinneer, Dominic Keating, John
           Billingsley, Jolene Blalock, Linda Park, Scott Bakula

   Director: Michael Grossman

   Executive Producers: Brannon Braga, Rick Berman

   Guest Stars: Paul Eliopoulos, Sean McGowan

   Writers: Andre Bormanis, Mike Sussman

   Original Air Date: Wed Feb 25 2004

   Genres: Adventure, Fantasy, Science fiction

   Airings:
      Sat Mar  5  7:00 PM -  8:00 PM on 8775 WSBK    (R, CC)
      Sun Mar  6  4:00 PM -  5:00 PM on 8105 WWOR    (R, CC)
      Sun Mar  6  8:00 PM -  9:00 PM on 8225 KBHK    (R, CC)

More information about the possible search terms can be found in the Query Language section below.

Prerequisites

To use xtvd-tools, you need the following:

MySQL 8.0 or later or MariaDB 10.2 or later
Python 3.7 or later
Python package mysqlclient

In addition, you will need to set up a Schedules Direct account to download TV listing data. For more information about this, see the installation instructions below.

Installation

Install MySQL 8.0 or later from http://www.mysql.com, or MariaDB 10.2 or later from https://mariadb.com or using whatever equivalent packaging tools come with your system.

Create a new MySQL database to hold the XTVD listing information. Run the following commands to create the database and set up a user which can be used to access it.

    % mysql -u root -p mysql
    Enter password: 

    mysql> CREATE DATABASE xtvd;
    mysql> CREATE USER xtvd_app@localhost IDENTIFIED BY 'xtvd_pw';
    mysql> GRANT ALL ON xtvd.* TO xtvd_app@localhost;
    mysql> FLUSH PRIVILEGES;
    mysql> QUIT;

Install Python 3.7 or later from http://www.python.org or your local packaging system. You'll also need the following Python package:
- mysqlclientdb from https://pypi.org/project/mysqlclient/
Visit http://www.schedulesdirect.org and click on the "Sign Up" link there to create a Schedules Direct account to use to download your TV listings. Enter your e-mail address and select a password and fill out the othe requested information. When asked for the software you use, select xtvd-tools. You will then need to create one or more lineups which contain the stations you wish to download listings for.
Once you have created your Schedules Direct account, create a file named .xtvd in your home directory of the following form, substituting in your Schedules Direct login name and password for username and password. Since this file contains your password, you may also wish to do a chmod 600 .xtvd on it to make it readable only to you.
```
    [auth]
    username: login_name
    password: password
```

Download, unpack, and install xtvd-tools-2.0.tar.gz by running the following commands:

    % tar zxf xtvd-tools-2.0.tar.gz
    % cd xtvd-tools-2.0
    % python3 setup.py install

Download your initial set of listing data by running the following command. You should see output similar to the following:

    % xtvd_load
    # 2005/02/26 14:11:54 Creating database...
    # 2005/02/26 14:11:55 Parsing version 1.3 data from 2005/02/26 to 2005/03/11
    # 2005/02/26 14:11:55 Found 147 stations
    # 2005/02/26 14:11:55 Parsing lineup DISH San Francisco
    # 2005/02/26 14:11:55 Found 147 channel mappings
    # 2005/02/26 14:12:30 Found 52011 schedules 
    # 2005/02/26 14:12:39 Found 18680 programs 
    # 2005/02/26 14:13:02 Found 87736 cast/crew members 
    # 2005/02/26 14:13:10 Found 31984 genres 
    # 2005/02/26 14:13:10 Adding secondary database indexes...
    # 2005/02/26 14:13:16 Database load complete

That's it! You should now be able to run xtvd_query on your newly imported listing data!

Query Language

The xtvd_query program searches TV listing information and prints the matching program and schedule information. The program can accept either a single query on its command line or a series of query commands from standard input. A command-line query is treated as a single 'add' command in the query language.

The query language allows you to execute a series of commands to build up a set of results by adding matching records from the current XTVD database or removing previously matched records from the current result set that meet specific criteria. Once the entire set of commands has run, the program will print out information about the programs and air times remaining in the result set. Commands are as follows:

add query_terms
Add records from the database matching the specified query terms into the result set.
remove query_terms
Remove records in the result set matching the specified query terms.

Multiple query terms are separated by spaces, and may optionally be preceded by "not" to negate a term. Values should be quoted with either single or double quotes if they contain spaces.

Terms doing string comparisons are case-insensitive exact matches on the corresponding field unless the term to be matched contains a '%' or '_'. In those cases, a SQL-style "LIKE" pattern match is done, where '%' matches zero or more of any character and '_' matches exactly one of any character. Available query terms are:

title title subtitle subtitle description description text text
These match on program title, subtitle (episode title), and program description. The term 'text' allows you to do a full-text match on words in any of these three fields. In this case, words in the query can appear in the entries in any order, with arbitrary other text around them.
date date_list time time_list day weekday_list
These match on the date, time of day, and day of the week a program begins airing. For the purposes of determining the date and day of week, days begin and end at 6:00am. So, a 3:00am Tuesday morning airing of a program is treated as airing on Monday. Dates are specified as 4-digit 'mmdd' values, with leading zeroes if appopriate. Times are specified as 4-digit 'hhmm' values in 24-hour time, with leading zeroes if appropriate. Days are expressed as any unique prefix of a weekday name. All of these terms accept a comma-separated list of values or ranges of values. Ranges are inclusive and expressed using '-' between the start and end elements of the range. So, for example, "Mon,Wed-Thu,Sat" is a valid list of days. Time ranges where the end time is smaller than the start time are treated as wrapping around midnight. Date ranges where the end date is smaller than the start date are treated as wrapping around the year end.
station station
This matches on the call sign of the station a program is airing on.
name person_name
This matches on the name of an actor, director, executive producer, guest star, host, producer, or writer in a program.
genre genre
This matches on the genre of the program, with values such as Action, Comedy, Drama, Science fiction, or Western.
movie series sports
These limit your match to programs marked as movies, TV series, or sporting events.
original
This matches on programs with an original air date less than two weeks before when the program airs, suggesting that the episode being aired is first run. This information is only available for episodes of TV series -- it will never match for movies. Like other terms, this can be prefixed by 'not' to negate the match.
cc ei hdtv mono new stereo subtitled
These match on the corresponding program characteristics. Like other terms, they can be prefixed by 'not' to negate the match. The term 'mono' is shorthand for 'not stereo' and vice-versa. Note that the 'new' flag is generally set for only the first airing of a show, making the 'original' query term above a better choice if you want to see all the airings which match a first run episode.
mpaaRating rating starRating rating tvRating rating
These match on the ratings a program received. MPAA and star ratings are generally only available for movies. MPAA rating values include G, PG, PG-13, R, NC-17, AO, and NR. Star ratings are from one to four stars and use '*' for full stars and '+' for half stars, so a star rating of '***+' would indictate three and a half stars. TV ratings are associated with particular airings of a program and values there include TV-Y, TV-Y7, TV-G, TV-PG, TV-14, and TV-MA.
year year_list
This matches on the specific year a movie was released. This information is only available for movies. Like date and time, this field allows a list of years and year ranges to be matched against.
partNumber part partTotal total_parts
For programs which are aired in multiple parts, these match on the part number and total number of parts of the program. In some cases, different parts will show up as different airings of the same program all sharing a common description, while in other cases each part may show up as a distinct program with its own description. However, either way, these terms allow you to match on specific part numbers.

Here are some examples of valid query commands:

    add day Mon-Fri time 1800 station KBHK

    add genre 'Science fiction' hdtv

    add starRating '****' year 1930-1949

    add text 'poker'

Here's an example of a sample query script, with a set of commands in it:

    # Find only first run episodes of these shows
    add original title 'Stargate: Atlantis'
    add original title 'The Dead Zone'

    # Find this show whenever it is on
    add title 'Roswell'

    # Find these actors whenever they are on, except as qualified
    add name 'Amber Tamblyn' not title 'Joan of Arcadia'
    add name 'Anna Paquin'
    add name 'Eliza Dushku' not title 'Angel' not title 'Buffy the Vampire Slayer'
    add name 'Jewel Staite'

    # Remove the following shows which would otherwise match the above
    remove title 'Buffy the Vampire Slayer' subtitle 'All the Way'
    remove title 'The Piano'
    remove title 'Widow on the Hill'

Support

This package is provided "AS IS", with no promise of support. However, I'd love to hear any comments you have about it and I'll do my best to answer any questions. Feel free to contact me at:

Ron Frederick <ronf@timeheart.net>

Page maintained by Ron Frederick