Table of Contents
XBMC ROM utilities (XRU) is a set of Python scripts that allows you to filter your ROM collections to remove unwanted ROMs from XBMC's launcher plugins like Advanced Launcher, ROM Collection Browser, or any other front-end. If the ROM collections are updated or a new version of the emulators installed (which may require some new ROMs to be updated) XRU is able to synchronise the ROM directories of your HTPC quickly and effectively.
XRU includes three programs
xru-console filters a ROM collection with No-Intro names and copies the filtered list to the destination directory of your choice. Multiple ROM collections, each with a different filter, can be configured. Multiple filters can be configured to the same ROM collection, for example to split the ROMs into USA, Europe and Japan regions. Also, if artwork is available, it may also be synchronised.
xru-mame takes the MAME-generated XML game database and merges it
with Catver.ini
. Then, a set of filters can be defined (for example,
to remove mechanical games or select games that use a lightgun) and filtered
ROMs are copied to different destination directories. It also creates NFO files
with game information for launchers and copies local artwork if available so
offline scrapers can be used.
xru-launcher-AL parses XBMC's Advanced Launcher configuration file and scans your ROM launchers for missing ROMs. It prints a report with the launchers you have to update.
xru-console requires a mandatory command to
specify the main action to do. Several commands also require a
filterName
. Optionally, some command actions can be
modified with options.
$ xru-console.py [-options] command [filterName]
The following commands are supported.
usage
Prints a summary of the commands and options.
list
Lists every ROM set system defined in the configuration file and
some basic information. Use --verbose
to get more
information.
list-nointro <filterName>
Lists every ROM defined in the No-Intro DAT file.
check-nointro <filterName>
Scans the source directory and reads the No-Intro XML data file. Checks if you have all the ROMs defined in the No-INTRO DAT and reports the number of missing and unknown ROMs.
list-tags <filterName>
Scans the source directory and reports the total number of ROM files, all the tags found, and the number of ROMs that have each tag. It also display ROMs without any tag.
check-filter <filterName>
Applies ROM filters defined in the configuration file and prints a list of the scored ROMs. used.
copy <filterName>
Applies the ROM filters defined in the configuration file and copies the contents of the source directory into the destination directory. This always overwrites ROMs in the destination directory and may take a long time for huge ROM collections and network-mounted destination directories.
update <filterName>
Like copy, but only copies ROMs if file sizes are different. This may save a lot of time.
check-artwork <filterName>
Reads the ROMs in destDir, checks if you have the corresponding artwork files, and prints a report.
copy-artwork <filterName>
Reads the ROMs in destDir and tries to copy the matching artwork to the destination directory.
update-artwork <filterName>
Like copy-artwork, but only copies images if file sizes are different. This may save a lot of time.
The following options are supported.
-h, --help
Prints a short command reference
-v, --verbose
Prints more information about what's going on. Can be used several times to get even more information.
-l, --log
Saves program output in xru-console-log.txt
.
--logto <logName>
Saves program output in the file you specify.
--dryRun
don't modify the destination directory at all, just print the operations to be done.
--cleanROMs
Deletes ROMs in the destination directory not present in the filtered ROM list (unknown ROMs).
--cleanNFO
Deletes redundant NFO files in destination directory (unknown NFO files).
--cleanArtWork
Deletes thumbs and fanart in the artwork destination directories not present in the artwork list (unknown Artwork).
xru-console requires a configuration file,
named xru-console-config.xml
, where you store the
directories and filters for every ROM set you want to filter. The configuration
filter is an XML file. A typical configuration filter looks like this.
<!-- Example configuration file for XBMC ROM utilities --> <ROMcollections> <!-- Nintendo ROMS --> <collection name="SNES NoIntro" shortname="snes"> <NoIntroDat>/home/xbmc/roms/DATs/SNES Parent-Clone (20140608-202038).dat</NoIntroDat> <ROMsSource>/home/xbmc/roms/roms-nintendo-snes/</ROMsSource> <ROMsDest >/home/xbmc/NUC-remote/ROMs/roms-nintendo-snes/</ROMsDest> <ThumbsSource>/home/xbmc/roms/ArtWork/nintendo-snes-boxes/</ThumbsSource> <FanartSource>/home/xbmc/roms/ArtWork/nintendo-snes-snaps/</FanartSource> <ThumbsDest >/home/xbmc/NUC-remote/ROMs/ArtWork/nintendo-snes-thumbs/</ThumbsDest> <FanartDest >/home/xbmc/NUC-remote/ROMs/ArtWork/nintendo-snes-fanart/</FanartDest> <filterUpTags>Europe, World, Rev 1</filterUpTags> <filterDownTags>Japan, Beta, Proto, France</filterDownTags> <includeTags>Europe, USA</includeTags> <excludeTags>Japan</excludeTags> </collection> <!-- Sega ROMS --> <collection name="Sega Mega Drive NoIntro" shortname="genesis"> <ROMsSource>/home/xbmc/roms/roms-sega-genesis/</ROMsSource> <ROMsDest >/home/xbmc/NUC-remote/ROMs/roms-sega-genesis/</ROMsDest> <filterUpTags>Europe, World, Rev 1</filterUpTags> <filterDownTags>Japan, Beta, Proto</filterDownTags> <includeTags>Europe, USA</includeTags> <excludeTags>Japan</excludeTags> </collection> </ROMcollections>
Every ROM collection filter requires a <collection>
tag. The attribute shortname
defines the name of this
collection to be used with the commands that require a <filterName>
.
name
attribute defines a longer name to better describe
this ROM collection filter and it is useful if you setup different filters
for the same ROM set.
<ROMsSource>
defines the source directory
where you have all the ROMs in your collection.
<ROMsDest>
defines the destination directory
where you filtered ROMs will be copied. Both tags are mandatory for every
ROM collection filter.
To use xru-console, your ROM collection needs
to have the No-Intro naming conventions. Only ROMs in
.zip
are currently supported.
If you have a ROM collection with different naming (TOSEC, Goodtools, or completely random names) then you can rename your ROMs with a ROM manager like CLRMamePro or ROM Vault.
For example, a (very abridged) No-Intro collection for the Super Nintendo may look like this.
Super Mario All-Stars and Super Mario World (Europe).zip Super Mario All-Stars + Super Mario World (USA) Super Mario World (Europe) (Rev 1).zip Super Mario World (Europe).zip Super Mario World (Japan) (En) (Arcade) [b].zip Super Mario World (USA).zip Super Mario - Yossy Island (Japan) (Rev 1).zip Super Mario - Yossy Island (Japan) (Rev 2).zip Super Mario - Yossy Island (Japan).zip
Every ROM in the collection has a base-name and then one or
more tags. Every single tag is inside parenthesis and the most common
tags are the region where the ROM belongs:
Europe
, USA
and/or Japan
. Some ROMs may have complex tags, separated
with commas, as in the following example.
Super Solitaire (Europe) (En,Fr,De,Es,It) (Proto).zip Super Solitaire (USA) (En,Fr,De,Es,It).zip
xru-console is able to remove all the tags from the ROM name and complex tags are also separated into simpler tags.
For every game in the ROM collection there are usually several ROMs, mainly released in different regions (Europe, USA, or Japan). In your XBMC launcher ideally you only want one ROM for every game, corresponding to the ROMs in your favourite region (or regions). Also, there may be several released version of your ROM and you just want to play the best available one.
xru-console makes two passes when filtering the ROMs in the source directory. Firstly, tags are removed from the ROM name, and all ROMs that have a common name after tag removal are grouped together. This is roughly like grouping ROMs into a parent/clone set. Note that the order of the ROMs in every set is arbitrary. For example, the SNES ROMs in the example above will be grouped like this.
<pcloneList> "Super Mario All-Stars and Super Mario World" Super Mario All-Stars and Super Mario World (Europe).zip | (Europe) <pcloneList> "Super Mario All-Stars + Super Mario World" Super Mario All-Stars + Super Mario World (USA).zip | (USA) <pcloneList> "Super Mario World" Super Mario World (Europe) (Rev 1).zip | (Europe), (Rev 1) Super Mario World (Europe).zip | (Europe) Super Mario World (Japan) (En) (Arcade) [b].zip | (Japan), (En), (Arcade) Super Mario World (USA).zip | (USA) <pcloneList> "Super Mario - Yossy Island" Super Mario - Yossy Island (Japan) (Rev 1).zip | (Japan), (Rev 1) Super Mario - Yossy Island (Japan) (Rev 2).zip | (Japan), (Rev 2) Super Mario - Yossy Island (Japan).zip | (Japan)
The first pass of the filtering process uses the configured labels
<filterUpTags>
and
<filterDownTags>
. For every tag you write inside
<filterUpTags>
, separated with commas, every ROM on
every parent/clone set will get some points. For example, if you write
this.
<filterUpTags>Europe, World, Rev 1</filterUpTags>
Then ROMs having the Europe tag will get 3 points, ROMs having the World tag will get 2 points, and ROMs having the Rev 1 tag will get one point. If a ROM has several wanted tags then it will get more points. Note that the tags you want more should be placed first.
The <filterDownTags>
filters works in a similar
way, but instead of adding points to the score it subtracts points. For
example, the following filter.
<filterDownTags>Japan, Beta, Proto, France</filterDownTags>
Will subtract -4 points for ROMs having the tag Japan, -3 points for the ROMs having the tag Beta, and so on. In the case of this filter, you specify ROMs you don not want first. Following with the example, if the filters are applied to our ROM list, then the scores will be.
<pcloneList> "Super Mario All-Stars and Super Mario World" 3 | Super Mario All-Stars and Super Mario World (Europe).zip | (Europe) <pcloneList> "Super Mario All-Stars and Super Mario World" 0 | Super Mario All-Stars + Super Mario World (USA).zip | (USA) <pcloneList> "Super Mario World" 4 | Super Mario World (Europe) (Rev 1).zip | (Europe), (Rev 1) 3 | Super Mario World (Europe).zip | (Europe) 0 | Super Mario World (USA).zip | (USA) -4 | Super Mario World (Japan) (En) (Arcade) [b].zip | (Japan), (En), (Arcade) <pcloneList> "Super Mario - Yossy Island" -2 | Super Mario - Yossy Island (Japan) (Rev 1).zip | (Japan), (Rev 1) -3 | Super Mario - Yossy Island (Japan) (Rev 2).zip | (Japan), (Rev 2) -3 | Super Mario - Yossy Island (Japan).zip | (Japan)
After scoring the ROMs, the first pass of the filtering takes the ROM with highest score on every set. If more than one ROM have the same highest score, then the first ROM in the list is selected. Following with the example, after the first filter the ROM collection will look like this.
<pcloneList> "Super Mario All-Stars and Super Mario World" 3 | Super Mario All-Stars and Super Mario World (Europe).zip | (Europe) <pcloneList> "Super Mario All-Stars and Super Mario World" 0 | Super Mario All-Stars + Super Mario World (USA).zip | (USA) <pcloneList> "Super Mario World" 4 | Super Mario World (Europe) (Rev 1).zip | (Europe), (Rev 1) <pcloneList> "Super Mario - Yossy Island" -2 | Super Mario - Yossy Island (Japan) (Rev 1).zip | (Japan), (Rev 1)
Out of a total of 9 ROMs, so far only 4 will be copied into the destination directory.
The second part of the filtering process allows you to remove unwanted ROMs with specific tags. Users in Europe or USA normally do not want Japanese ROMs at all. To remove ROMs belonging to the Japanese region configure.
<includeTags>Europe, USA</includeTags> <excludeTags>Japan</excludeTags>
This filter will exclude all ROMs with the tag Japan, but only if the ROM has not the tags Europe or USA. This is to prevent filtering ROMs with tags like (Japan, USA) or (Japan, Europe). After the second pass of the filtering process, the filtering is over and the list of ROMs to be copied to the destination directory will look like this.
<pcloneList> "Super Mario All-Stars and Super Mario World" 3 | Super Mario All-Stars and Super Mario World (Europe).zip | (Europe) <pcloneList> "Super Mario All-Stars and Super Mario World" 0 | Super Mario All-Stars + Super Mario World (USA).zip | (USA) <pcloneList> "Super Mario World" 4 | Super Mario World (Europe) (Rev 1).zip | (Europe), (Rev 1)
The list of tags for every ROM collection can be displayed with the
list-tags
command, which also prints how many ROMs
in the collection have that tag. This command is very handy to know which tags
to put in the configuration file for that ROM collection. Note that if a ROM
has more than one tag it will be counted twice. For example:
$ ./xru-console.py list-tags snes XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Listing tags] Filter name = snes [Tag histogram] ... 104 Fr 120 En 173 Rev 1 191 BS 220 Beta 626 Europe 968 USA 1925 Japan
Before you copy/update your filtered ROM collection, you may
want to check what will happen and the check-filter
command does precisely this. For example:
$ ./xru-console.py check-filter snes XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Check-filter ROM] Filter name = snes Using No-Intro parent/clone DAT Parsing No-Intro XML DAT Parsing No-Intro XML file ./Nintendo - SNES Parent-Clone (20140608-202038).dat... done Total number of games = 3366 Number of parents = 1828 Number of clones = 1538 [Filtering ROMs] ... <ROM set> Super Mario All-Stars + Super Mario World 0 [IH] Super Mario All-Stars + Super Mario World (USA).zip <ROM set> Super Mario All-Stars and Super Mario World 7 [IH] Super Mario All-Stars and Super Mario World (Europe).zip ... <ROM set> Super Mario Kart 7 [IH] Super Mario Kart (Europe).zip 0 [IH] Super Mario Kart (USA).zip -3 [EH] Super Mario Kart (Japan).zip <ROM set> Super Mario RPG -3 [EH] Super Mario RPG (Japan).zip <ROM set> Super Mario RPG - Legend of the Seven Stars 0 [IH] Super Mario RPG - Legend of the Seven Stars (USA).zip ...
Have a look at the end of next section for a quick explanation of the flags and numbers in this listing.
The first pass of the filtering process groups different versions of the same game together. However, based only on the ROM names this process is far from perfect and will produce wrong results for some games. For example:
<pcloneList> "Super Mario All-Stars and Super Mario World" Super Mario All-Stars and Super Mario World (Europe).zip | (Europe) <pcloneList> "Super Mario All-Stars + Super Mario World" Super Mario All-Stars + Super Mario World (USA).zip | (USA)
Will be grouped into different parent/clone sets, however it is clear that they are just the European and USA version of the same game. Actually, this example (however real) is not very common. Most of the games belonging to the same set having different name are the Japanese versions of the game.
For many users, the basic filtering will be fine. However, for picky users this problem may be solved using the No-Intro One ROM-One Game (1G1R) DATs. A typical No-Intro DAT looks like this.
game ( name "Sonic The Hedgehog (Japan, Korea)" description "Sonic The Hedgehog (Japan, Korea)" rom ( name "Sonic The Hedgehog (Japan, Korea).md" size 524288 crc AFE05EEE ) ) game ( name "Sonic The Hedgehog (USA, Europe)" description "Sonic The Hedgehog (USA, Europe)" rom ( name "Sonic The Hedgehog (USA, Europe).md" size 524288 crc F9394E97 flags verified ) )
However, from DAT-o-MATIC
a No-Intro DAT in XML format having proper parent/clone lists are available. This is also
referred to as a 1G1R list. To download a 1G1R XML DAT for your favourite system,
go to DAT-o-MATIC, click
on the P/Clone XML
link, select your console system
in the drop-box on the top, and finally click on the
Download
button. This is how a 1G1R XML DAT looks like.
<game name="Sonic The Hedgehog (USA, Europe)"> <description>Sonic The Hedgehog (USA, Europe)</description> <release name="Sonic The Hedgehog" region="EUR"/> <release name="Sonic The Hedgehog" region="USA"/> <rom name="Sonic The Hedgehog (USA, Europe).md" size="524288" crc="F9394E97" status="verified"/> </game> <game name="Sonic The Hedgehog (Japan, Korea)" cloneof="Sonic The Hedgehog (USA, Europe)"> <description>Sonic The Hedgehog (Japan, Korea)</description> <release name="Sonic The Hedgehog" region="JPN"/> <release name="Sonic The Hedgehog" region="KOR"/> <rom name="Sonic The Hedgehog (Japan, Korea).md" size="524288" crc="AFE05EEE"/> </game>
If you pay attention, there is a cloneof
attribute
on every clone ROM in the collection. This allows
xru-console to make a perfect parent/clone list
before applying filters, and this will produce a better filtered list.
In order to use a No-Intro 1G1R XML DAT with xru-console, you need to configure the following option for every collection filter where you want to use it. Make sure you use the correct XML DAT for the collection!
<collection> ... <NoIntroDat>./Sega - Genesis Parent-Clone (20140601).dat</NoIntroDat> ... </collection>
In No-Intro 1G1R mode, xru-console makes the main parent/clone list using the XML DAT instead of scanning you source directory. Then, the two pass filtering is applied. Finally, your source directory will be scanned and for every ROM set in the parent/clone list, the ROM with highest score will be copied into the destination directory. If you do not have that ROM, then the second ROM in the scored ROM set will be tried, and so on.
In No-Intro 1G1R mode, two additional commands are available.
list-nointro <romSetName>
lists all the games in the
No-Intro 1G1R XML DAT. For example:
$ ./xru-console.py list-nointro genesis XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Listing No-Intro XML DAT] Filter name = genesis Parsing No-Intro XML DAT Parsing No-Intro merged XML file ./Sega - Genesis Parent-Clone (20140601).dat... done ... <game> Sonic The Hedgehog (USA, Europe) <game> Sonic The Hedgehog (Japan, Korea) <game> Sonic The Hedgehog 2 (World) (Rev A) <game> Sonic The Hedgehog 2 (World) <game> Sonic The Hedgehog 2 (World) (Beta) <game> Sonic The Hedgehog 3 (Europe) <game> Sonic The Hedgehog 3 (Japan, Korea) <game> Sonic The Hedgehog 3 (USA) ... <game> Zoop (USA) Number of games = 1681
check-nointro <romSetName>
will scan you source
directory and will report how many ROMs you have, how many ROMs you don't
have (missing ROMs), and how many ROMs you have not listed in the
DAT (unknown ROMs).
$ ./xru-console.py check-nointro genesis-local-nointro XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Checking ROMs with No-Intro XML DAT] Filter name = genesis-local-nointro Parsing No-Intro XML DAT Parsing ./DAT-No-Intro/Sega - Genesis Parent-Clone (20140601-111936).dat... done [Scanning ROMs in sourceDir] [Report] Games in DAT = 1681 Have ROMs = 1681 Missing ROMs = 0 Unknown ROMs = 47
If you specify the option -v
,
xru-console will print the missing and unknown ROMs.
-vv
will also print the ROMs you have.
Note that xru-console only looks at the file name of the ROM! It does not check the ROM checksum like more advanced ROM Managers, for example CRLMamePro or ROM Vault.
Before you copy/update your filtered ROM collection, you may
want to check what will happen and the check-filter
command does precisely that. For example:
$ ./xru-console.py check-filter snes XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Check-filter ROM] Filter name = snes Using No-Intro parent/clone DAT Parsing No-Intro XML DAT Parsing No-Intro XML file ./Nintendo - SNES Parent-Clone (20140608-202038).dat... done Total number of games = 3366 Number of parents = 1828 Number of clones = 1538 [Filtering ROMs] ... <ROM set> Super Mario All-Stars and Super Mario World 7 [IH] Super Mario All-Stars and Super Mario World (Europe).zip 0 [IH] Super Mario All-Stars + Super Mario World (USA).zip ... <ROM set> Super Mario RPG - Legend of the Seven Stars 0 [IH] Super Mario RPG - Legend of the Seven Stars (USA).zip -3 [EH] Super Mario RPG (Japan).zip <ROM set> Super Mario World 11 [IH] Super Mario World (Europe) (Rev 1).zip 7 [IH] Super Mario World (Europe).zip 0 [IH] Super Mario World (USA).zip -3 [EH] Super Mario World (Japan) (En) (Arcade) [b].zip -3 [EH] Super Mario World - Super Mario Bros. 4 (Japan).zip ...| || | |+-- Have flag | +--- Included flag +------ Score
In this ROM listing, you can see the parent/clone sets, the score of each ROM, if the ROM is included or excluded, and if the ROM exists in the source directory or not. Note that if you use a No-Intro XML DAT you may have missing ROMs. However, if you do not use the DAT then you will always have all the ROMs in this list.
Once you have setup your configuration file and checked the filter output (note that you can configure several filters for the same ROM source directory, for example, to place Europe/USA games and Japanese game into different destination directories) then it is time to copy your filtered ROMs from the source directory into the destination directory.
The copy <romSetName>
command will filter
your ROMs and then copy, one by one, every filtered ROM into the
destination directory.
$ ./xru-console.py copy genesis XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Copy/Update ROMs] Filter name = genesis-local Using directory listing [Reading ROMs in source dir] Found 1728 ROMs [Filtering ROMs] [Scanning sourceDir for ROMs to be copied] [Creating list of ROMs to be copied/updated] [Copying ROMs into destDir] 0% <Copied> 10 Super Jogos (Brazil).zip 0% <Copied> 16 Zhang Ma Jiang (China) (Unl).zip 0% <Copied> 3 Ninjas Kick Back (USA).zip ... 99% <Copied> [BIOS] Sega CD 2 (USA) (v2.00W).zip 99% <Copied> [BIOS] Sega TMSS (World).zip 99% <Copied> [BIOS] X'Eye (USA) (v2.00).zip [Report] Copied 962 ROMs
The update <romSetName>
command operates
in a similar way to copy
. However, rather than
copying all the ROMs, each ROMs in sourceDir is compared with the same
named ROM in destDir, and the file will be copied only if the destination
ROM does not exist or the file sizes are different. This is very
useful when you destination directory is network-mounted.
$ ./xru-console.py update genesis --cleanROMs XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Copy/Update ROMs] Filter name = genesis Using No-Intro parent/clone DAT Parsing No-Intro XML DAT Parsing No-Intro XML file ./Sega - Genesis Parent-Clone (20140601-111936).dat... done Total number of games = 1681 Number of parents = 1010 Number of clones = 671 [Filtering ROMs] [Scanning sourceDir for ROMs to be copied] [Creating list of ROMs to be copied/updated] [Updating ROMs into destDir] 0% <Copied > Action 52 (USA) (Alt 1) (Unl).zip 31% <Copied > Gain Ground (World) (Alt 1).zip 32% <Copied > Ghostbusters (World).zip 35% <Copied > HardBall III (USA) (Unl).zip ... 98% <Copied > Zany Golf (USA, Europe).zip 99% <Copied > [BIOS] LaserActive (USA) (v1.02).zip [Report] Copied ROMs 47 Updated ROMs 821 [Cleaning ROMs in ROMsDest] <Deleted> ATP Tour Championship Tennis (USA).zip <Deleted> Action 52 (USA) (Unl).zip ... <Deleted> [BIOS] Sega CD 2 (USA) (v2.00W).zip Deleted 122 redundant ROMs
By default, only the newly copied files are displayed during the update
process. If you want to check also the updated files, then use the
option -v
in the command line.
With both copy
and update
commands, you can use the flags --dryRun
,
--cleanROMs
and --cleanNFO
.
--dryRun
will not perform any operation in your
files. Use this flag if you want to test what would happened with your ROMs,
and if you are happy with the results execute
xru-console without --dryRun
option. Note that if you use --dryRun
,
--cleanROMs
and --cleanNFO
will
show you inaccurate outcomes, since the destination directory is not modified.
--cleanROMs
will delete any unknown ROMs in the
destination directory. After copying/update the filtered ROMs, the
destination directory will be scanned and any ROM found not in the list of
filtered ROMs will be deleted. Use this with caution! The list of deleted ROMs
in destDir is always printed.
Both Advanced Launcher
and ROM Collection Browser
create NFO files having the scrapped ROM information, like year of the release,
genre, etc. Use the --cleanNFO
to remove any unknown
NFO files, in a similar way to the --cleanROMs
flag. The
list of deleted NFO files in destDir is always printed.
Both Advanced Launcher and ROM Collection Browser have good scrapers that will download most of the artwork for your ROMs. However, the scraping process may be very slow for huge ROM collections (for example, a collection having 2,000 ROMs may take up to 10 hours to scrap). This problem may be solved if you have locally available collections of artwork. xru-console can copy the matching artwork for your filtered collection in the directories you specify, and then you can choose the offline scraper that simply reads those files from the disk instead of downloading them from the Internet.
xru-console currently uses two types of
artwork files, named thumbs
and
fanart
, which correspond with the two artwork categories
used by Advanced Launcher.
Roughly speaking, thumbs are images that will be displayed for every game in
a prominent position (this is XBMC skin and listing-mode dependant) and
fanart will appear as a background. You can think of the thumbs as the game box
and the fanart as the in-game screenshot.
Full collections of No-Intro ROMs exist out there. However, it is very difficult to find a complete artwork collection that have boxes/screenshots/titles. Partial artwork collections are available, although. To overcome this problem, xru-console will try to replace any missing piece of artwork with another one belonging to the same parent/clone set of the filtered ROM, in the case you have a partial artwork set which does not belong to the region of the ROMs you are filtering. We will see how this works later with an example.
If you have local Artwork available and want to copy it into destination folders (synchronised with your filtered ROM list), then configure the following into your ROM collection
<collection> ... <ThumbsSource>/home/xbmc/roms/ArtWork/nintendo-snes-boxes/</ThumbsSource> <FanartSource>/home/xbmc/roms/ArtWork/nintendo-snes-snaps/</FanartSource> <ThumbsDest >/home/xbmc/NUC/ROMs/ArtWork/nintendo-snes-thumbs/</ThumbsDest> <FanartDest >/home/xbmc/NUC/ROMs/ArtWork/nintendo-snes-fanart/</FanartDest> ... </collection>
You may check you configuration with the
check-artwork <romSetName>
command. For example:
$ ./xru-console.py check-artwork genesis XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Check-ArtWork] Filter name = genesis Using No-Intro parent/clone DAT Parsing No-Intro XML DAT Parsing No-Intro XML file ./Sega - Genesis Parent-Clone (20140601-111936).dat... done Total number of games = 1681 Number of parents = 1010 Number of clones = 671 [Optimising ArtWork file list] [Updating ArtWork] ... << ROM >> Sonic The Hedgehog (USA, Europe).zip Original Sonic The Hedgehog (USA, Europe) Have T Sonic The Hedgehog (USA, Europe).png Have F Sonic The Hedgehog (USA, Europe).png ... << ROM >> Speedball 2 (Europe).zip Replaced Speedball 2 - Brutal Deluxe (USA) Have T Speedball 2 - Brutal Deluxe (USA).png Have F Speedball 2 - Brutal Deluxe (USA).png ... << ROM >> [BIOS] X'Eye (USA) (v2.00).zip Not found [Report] Number of ROMs in destDir = 868 Number of ArtWork found = 722 Number of original ArtWork = 179 Number of replaced ArtWork = 543 Number of have Thumbs = 722 Number of missing Thumbs = 0 Number of have Fanart = 707 Number of missing Fanart = 15
Note that for the game Speedball 2 no artwork was found for the filtered game in destDir. However, a replacement belonging to another ROM in the parent/clone set was found. Some other ROMs, particularly BIOSes, won't have any artwork and will produce a miss.
When you are happy with the results, you can copy/update your
Artwork with the commands copy-artwork <romSetName>
and update-artwork <romSetName>
. You can use the
--cleanArtWork
flag if you want to delete
unknown Artwork into the thumbs and fanart destination directories. For example:
$ ./xru-console.py update-artwork genesis --cleanArtWork XBMC ROM utilities - Console ROMs version 0.1.0 [Parsing config file] [Updating/copying ArtWork] Filter name = genesis Using No-Intro parent/clone DAT Parsing No-Intro XML DAT Parsing No-Intro XML file ./Sega - Genesis Parent-Clone (20140601-111936).dat... done Total number of games = 1681 Number of parents = 1010 Number of clones = 671 [Optimising ArtWork file list] [Updating ArtWork] 1% <Missing Fanart> Adventurous Boy - Mao Xian Xiao Zi (China) (Unl) 5% <Missing Fanart> Awesome Possum Kicks Dr Machino's Butt! (USA) (Beta) ... 96% <Missing Fanart> World Series Baseball 98 (USA) [Report] Copied thumbs 0 Updated thumbs 722 Missing thumbs 0 Copied fanart 0 Updated fanart 707 Missing fanart 15 [Cleaning ArtWork] Deleted 0 redundant thumbs Deleted 0 redundant fanart
xru-mame is able to filter a MAME ROM collection. This tool is somewhat more sophisticated than xru-console because MAME keeps a full database with a lot of information about the arcade game it runs. This database can be used with a lot of filtering options to fully customize the list of games in your front-end.
xru-mame requires a command that defines the main action to do. Also, many commands require the name of the filter to use.
$ xru-mame.py [options] <command> [filterName]
The following commands are supported.
usage
prints a summary of the commands and options.
reduce-XML
Takes MAME XML as input, picks the useful information, and writes an stripped XML with only meaningful information. The reason for doing this is because MAME XML file is huge and takes a long time to process it. After reducing it, all subsequent processing should be much quicker.
merge
Takes MAME XML (reduced) info file and Catver.ini
and makes an output XML file with all the necessary information for proper
game filtering.
list-merged
List every ROM set system defined in the merged MAME XML information file.
Use --verbose
to get more information.
list-categories
Reads Catver.ini
and makes a histogram of the categories
(prints all available categories and tells how many ROMs every category has).
list-drivers
Reads merged XML database and prints a histogram of the drivers (how many games use each driver).
list-controls
Reads merged XML database and prints a histogram of the game controls: buttons, players and input devices.
list-years
Reads merged XML database and prints a histogram of the game release year (how many games were released on each year).
check-filter <filterName>
Applies filters and checks you source directory for have and missing ROMs.
copy <filterName>
Applies ROM filters defined in the configuration file and copies the contents of sourceDir into destDir. This overwrites ROMs in destDir.
update <filterName>
Like copy, but only copies files if file size is different (this saves a lot of time, particularly if sourceDir and/or destDir are on a network-mounted filesystem).
check-artwork <filterName>
Reads the ROMs in destDir, checks if you have the corresponding artwork files, and prints a report.
copy-artwork <filterName>
Reads the ROMs in destDir and tries to copy the matching artwork to the destination directory.
update-artwork <filterName>
Like copy-artwork, but only copies images if file sizes are different. This may save a lot of time.
The following options are supported.
-h, --help
Print short command reference
-v, --verbose
Print more information about what's going on
-l, --log
Save program output in xru-mame-log.txt
.
--logto [logName]
Save program output in the file you specify.
--dryRun
Don't modify destDir at all, just print the operations to be done.
--cleanROMs
Deletes ROMs in destDir not present in the filtered ROM list.
--generateNFO
Generates NFO files with game information for the launchers.
--cleanNFO
Deletes unknown NFO files in the destination directory.
--cleanArtWork
Deletes unknown ArtWork in destination directories.
xru-mame requires a configuration file,
named xru-mame-config.xml
,
where you setup the ROM directories and filters. A typical
configuration filter looks like this.
<!-- Example configuration file for XBMC ROM utilities --> <MAMEConfig> <General> <MAME_XML >./DAT-mame/mame-0154.xml</MAME_XML> <MAME_XML_redux>./DAT-mame/mame-0154-reduced.xml</MAME_XML_redux> <Catver >./DAT-mame/Catver.ini</Catver> <MergedInfo >./DAT-mame/mame-0154-merged.xml</MergedInfo> </General> <MAMEFilter name="main"> <ROMsSource>/home/xbmc/roms/roms-mame/</ROMsSource> <ROMsDest >/home/xbmc/NUC-remote/ROMs/roms-mame-main/</ROMsDest> <ThumbsSource>/home/xbmc/roms/ArtWork/nintendo-snes-boxes/</ThumbsSource> <FanartSource>/home/xbmc/roms/ArtWork/nintendo-snes-snaps/</FanartSource> <ThumbsDest >/home/xbmc/NUC-remote/ROMs/ArtWork/nintendo-snes-thumbs/</ThumbsDest> <FanartDest >/home/xbmc/NUC-remote/ROMs/ArtWork/nintendo-snes-fanart/</FanartDest> <MainFilter>NoClones, NoSamples, NoMechanical, NoBIOS, NoNonworking</MainFilter> <Driver>not cps1</Driver> <Categories>not Mature and not Casino and not PinMAME and not Fruit_Machines</Categories> <Controls>not Mahjong and not Gambling and not Hanafuda</Controls> <Buttons>buttons == 1 or buttons == 2</Buttons> <Players>players == 1 or players == 2</Players> <Years>year >= 1990 and year < 2000</Years> <!-- <YearsOpts>YearExpansion</YearsOpts> --> </MAMEFilter> <MAMEFilter name="cps1"> <ROMsSource>/home/xbmc/roms/roms-mame/</ROMsSource> <ROMsDest >/home/xbmc/NUC-remote/ROMs/roms-mame-cps1/</ROMsDest> <MainFilter>NoClones, NoSamples, NoMechanical</MainFilter> <Driver>cps1</Driver> </MAMEFilter> </MAMEConfig>
The first step before games can be filtered is to create a MAME game database that xru-mame can use. First, check what is your MAME version with the following command.
$ mame -? M.A.M.E. v0.154 (Jul 23 2014) - Multiple Arcade Machine Emulator Copyright Nicola Salmoria and the MAME team MAME is an emulator: it reproduces, more or less faithfully, the behaviour of ...
In this example MAME has the version number 0.154. This number will be used in the configuration files so when you get a new version of MAME you won't mess up game databases.
Next, create the MAME XML game database. Execute
the following command to place the XML database into mame-0154.xml
:
$ mame -listxml > mame-0154.xml
Now, create a basic xru-mame-config.xml
configuration
file. Note that in this example all the MAME related
game databases will be stored in the directory DAT-mame
placed where you have the xru-mame.py
script for
convenience.
<MAMEConfig> <General> <MAME_XML >./DAT-mame/mame-0154.xml</MAME_XML> <MAME_XML_redux>./DAT-mame/mame-0154-reduced.xml</MAME_XML_redux> <Catver >./DAT-mame/Catver.ini</Catver> <MergedInfo >./DAT-mame/mame-0154-merged.xml</MergedInfo> </General> </MAMEConfig>
MAME XML game database is huge and has much more
information than needed for game filtering. Next, the game database will be "reduced",
that is, non-relevant information will be stripped off and a shorted game database
will be created (named mame-0154-reduced.xml
in the above
configuration file example). Execute:
$ ./xru-mame.py reduce-XML XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Reducing MAME XML game database] Reading MAME XML game database... NOTE: this will take a looong time... Parsing MAME XML file ./DAT-mame/mame-0154.xml... done Reducing MAME XML database... Building reduced output XML file... Writing reduced XML file ./DAT-mame/mame-0154-reduced.xml
Please be patient! This reduction process will take several minutes and
about 2 GB of RAM will be used. However, all subsequent commands will be much
faster. The reduction will shrink mame-0154.xml
to
mame-0154-reduced.xml
from about 150 MB to about 15 MB.
Next, download the file Catver.ini
from
Progetto EMMA
(make sure you get the English version) and place it in the
DAT-mame
directory. The following command
will merge mame-0154-reduced.xml
with
Catver.ini
to create
mame-0154-merged.xml
and then your are done.
Execute:
XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Building merged MAME filter database] [Parsing Catver.ini] [Parsing (reduced) MAME XML file] Parsing MAME XML file ./DAT-mame/mame-0154-reduced.xml... done [Merging MAME XML and categories] [WARNING] Category not found for game 18wheelu ... [WARNING] Category not found for game r4700le [Writing output file] Output file ./DAT-mame/mame-0154-merged.xml
If you get some some warnings about non-found categories for some games just ignore them. You can now list the games in your merged XML database.
$ ./xru-mame.py list-merged XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Short listing of reduced MAME XML] Parsing merged MAME XML file './DAT-mame/mame-0154-reduced-merged.xml'... done 005 |-- driver = segag80r.c |-- sampleof = 005 |-- description = 005 |-- year = 1981 |-- manufacturer = Sega |-- driver status = imperfect +-- category = Maze ... keycus_c430 |-- driver = src/mame/machine/ns11prot.c |-- isdevice = yes |-- description = KEYCUS C430 +-- category = System [Report] Number of games = 30551 Number of clones = 20513 Number of games with samples = 573 Number of devices = 907
You are now ready to start filtering your MAME ROMs.
In order to understand how the MAME filters
work, let's try a basic filter named main. Edit
your xru-mame-config.xml
and add the following.
<MAMEConfig> <General> <MAME_XML >./DAT-mame/mame-0154.xml</MAME_XML> <MAME_XML_redux>./DAT-mame/mame-0154-reduced.xml</MAME_XML_redux> <Catver >./DAT-mame/Catver.ini</Catver> <MergedInfo >./DAT-mame/mame-0154-merged.xml</MergedInfo> </General> <MAMEFilter name="main"> <ROMsSource>/home/wintermute/mame-roms/</ROMsSource> <ROMsDest >/home/NUC-remote/ROMs/roms-mame-main/</ROMsDest> <MainFilter>NoClones, NoSamples, NoMechanical, NoBIOS, NoNonworking</MainFilter> <Driver>not cps1 and not cps2 and not cps3 and not neogeo_noslot</Driver> <Categories>not Casino and not PinMAME and not Fruit Machines</Categories> <Controls>Joy or Lightgun and not(Mahjong or Hanafuda)</Controls> <Buttons>buttons == 1</Buttons> <Players>players == 1</Players> <Years>year == 1990 or year == 1991</Years> </MAMEFilter> </MAMEConfig>
Every MAME filter requires a
<MAMEFilter>
tag and there the name
attribute
defines the name you will used to invoke other command for this filter. Every
filter require the <ROMsSource>
and
<ROMsDest>
tags, which define the source
directory when you have all your ROMs and the destination directory where
filtered ROMs will be copied.
The rest of the tags inside MAMEFilter
allows to
define filters to discard the games you don not want and keeps the ones you
want.
The first filter is configured with the tag
<MainFilter>
. Here you can write the following
filtering options (separated by commas if more than one).
NoClones
will filter all the cloned games.
Clones are derived games, maybe in a different language other than
English or bootleg games. Most users will choose this option.
NoSamples
will filter all the games that use samples.
NoMechanical
will remove all the mechanical games. Most
of them are slot or pinball machines and do not work anyway.
NoBIOS
will remove all BIOS ROMs. BIOS are non-playable
ROMs necessary for other ROMs to work. Required BIOS are automatically added
as dependencies (that is, if a game you want needs a specific BIOS or device
they will be automatically included regardless of your filters), so using
this option is always safe and recommended.
NoNonworking
will remove all games that do not work
or have serious emulation problems or emulation is preliminary.
Note that games belonging to the devices
type are removed automatically. Those are not really games but
emulated devices and are non-playable. If a ROM you want requires some
device or BIOS these will be automatically added as dependencies so
removing devices (and BIOS) is a safe operation.
After the main filter comes the driver filter, which is defined
inside the tag <Driver>
. By default all drivers
will be included. If you specify a driver, then only games belonging
to that driver will be included. The not
operator
in front of a driver name excludes games belonging to that driver.
You can also specify multiple drivers separated by and
and or
operators and use parenthesis to change the
operator precedence. For example:
(a) <Driver>cps1</Driver> (b) <Driver>not cps1</Driver> (c) <Driver>not cps1 and not cps2</Driver> (d) <Driver>not (cps1 or cps2)</Driver> (e) <Driver>cps1 or cps2</Driver> (a) Include CPS1 games. (b) Exclude CPS1 games. (c) Exclude both CPS1 and CPS2 games. (d) Same as (c). Just another way of writing this filter. (e) Include CPS1 or CPS2 games.
You can list the MAME drivers with
the command list-drivers
.
$ ./xru-mame.py list-drivers XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Listing MAME drivers] NOTE: clones are not included NOTE: mechanical are not included NOTE: devices are not included Parsing merged MAME XML file './DAT-mame/mame-0154-merged.xml'... done [Final (used) drivers] 1 calorie.c 1 subs.c ... 157 neogeo_noslot.c 160 peplus.c 199 naomi.c 471 bfm_sc4.c
Note that you do not need to write the trailing .c when configuring driver filters.
Categories are controlled with the tag
<Categories>
. This filter and the filter
language (and
, or
,
not
, (
, )
)
works in a similar way to the <Driver>
filter,
but instead of drivers you specify game categories. For example:
(a) <Categories>not Casino and not PinMAME and not Fruit_Machines</Categories> (b) <Categories>Fighter or Shooter</Categories> (c) <Categories>Fighter and Shooter</Categories> (a) Exclude games belonging to Casino, PinMAME or Fruit_Machines. (b) Include only Fighter or Shooter games. (c) Will exclude all games. Note that every game has a unique category, so no game can be a Fighter AND a Shooter at the same time. Be careful when configuring your filters.
You can have a look at the available categories with
the list-categories
command.
$ ./xru-mame.py list-categories XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Listing categories from Catver.ini] Opening ./DAT-mame/Catver.ini [Making categories histogram] [Final (used) categories] 20 Climbing ... 1342 PinMAME 1343 Fighter 1761 Casino 2496 Shooter 16779 Fruit Machines
Note that Catver.ini
defines a main category and
a sub-category for each game. For example:
[Category] 005=Maze / Shooter Small 10yard=Sports / Football Amer. 10yard85=Sports / Football Amer. 10yardj=Sports / Football Amer. ... | | | +- Sub-category +---------- Main category
xru-mame only takes into account the main category and ignores any sub-categories. For example, the game 005 will have category Maze, 10yard Sports, etc. Also, when merging the categories into the merged XML database some categories are renamed for convenience, spaces are changed with _, and dots are removed. For example:
System / BIOS --> BIOS Electromechanical - PinMAME --> PinMAME Ball & Paddle --> Ball_and_Paddle
Any game that contains the category *Mature* either in the main
category or in the sub-category will be assigned to the category Mature
(no * in the final category name). In addition to the renamed, final categories
you can see the original Catver.ini
categories with the
-v
switch and even more
information with -vv
.
With the controls filter you can choose games based on the input peripherals they require. This is useful if you only have a digital joystick and will no be able to play games that require a Lightgun. Also, it allows you to remove Mahjong games and fruit machines (fruit machines typically have the controls ButtonsOnly).
The controls filter is configured with the tag
<Controls>
. This filter and the filter
language (and
, or
,
not
, (
, )
)
works in a similar way to the <Driver>
and
<Categories>
filters. For example:
(a) <Controls>Joy</Controls> (b) <Controls>Joy and not Mahjong</Controls> (c) <Controls>not Mahjong and not Hanafuda</Controls> (d) <Controls>not (Mahjong or Hanafuda)</Controls> (e) <Controls>Joy and not(Mahjong or Hanafuda)</Controls> (a) Include if Joy and maybe other controls. (b) Include if Joy and maybe other controls, but not Mahjong. (c) Exclude game if Mahjong or Hanafuda. (d) Same as above (e) Include games that use digital Joystick and exclude games which use Mahjong or Hanafuda controls.
You can have a look at the available controls with
the list-controls
command.
$ ./xru-mame.py list-controls XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Listing MAME controls] NOTE: clones are not included NOTE: mechanical are not included NOTE: devices are not included Parsing merged MAME XML file './DAT-mame/mame-0154-reduced.xml'... done [Input - control - type histogram (per game)] 1 joy, mahjong, stick 1 keypad, stick ... 487 hanafuda, mahjong 519 gambling 1434 ButtonsOnly 2874 joy [Input - buttons histogram] 9 14 13 13 ... 1178 0 1328 2 [Input - players histogram] 1 6 2 8 ... 2704 1 2990 2 [Input - control - type histogram] 9 Keyboard 14 Keypad ... 776 Mahjong 1434 ButtonsOnly 3134 Joy
This command additionally tells you the buttons and player
statistics. Note that some games require more than one control, and this is
indicated in the
[Input - control - type histogram (per game)]
.
However, in the [Input - control - type histogram]
games that use two controls are counted twice, three controls are counted
three times, and so on.
This filter allows you to filter games based on the number of buttons
that they require. This filter works as a Python expression and all
Python operators can be used with the variable buttons
.
Let's see this with an example. For example:
(a) <Buttons>buttons == 1 or buttons == 2</Buttons> (b) <Buttons>buttons <= 8</Buttons> (a) Include games with 1 or 2 buttons. (b) Include games that require 8 buttons or less. The operator <= must be written <= in the configuration file because of the XML rules.
You can see the button statistics with the list-controls
command. Have a look at the previous section for an example.
This filter allows you to select games based on the number of players. Note
that a game for 2 players means a game in which the two players may play simultaneously
or alternated. The players filter is a Python expression over the variable
players
, for example:
(a) <Players>players == 1 or players == 2</Players> (a) Include games with 1 or 2 players.
You can see the player statistics with the list-controls
command. Have a look at the controls filter section for an example.
With this filter you can choose games based on the year they were released. This is very useful to get rid of the oldies or to configure the 1980s launcher, the 1990s launcher, etc.
This filter works in a similar way to the Buttons
and Players
filters. Here, you can filter the
variable year
. For example:
(a) <Years>year == 1982</Years> (b) <Years>year >= 1990 and year < 2000</Years> (a) Include all games released in 1982. (b) Include games released between 1990 and 1999.
You can see the game release statistics with the
list-years
command.
$ ./xru-mame.py list-years XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Listing MAME controls] NOTE: clones are not included NOTE: mechanical are not included NOTE: devices are not included Parsing merged MAME XML file './DAT-mame/mame-0154-reduced.xml'... done [Release year histogram (raw)] 1 2007? 1 1973 ... 2 ???? ... 5 197? ... 261 1996 345 1995 597 200? 668 199? [Release year histogram (trimmed)] 22 2011 23 2010 ... 952 1996 1035 1995
Most games have a well determined release year, for example 1996. However,
some games (most of them fruit machines) have release years of the form 199?.
Also, for other games the release year is not completely determined, for example
1998?, or even not known at all (????). xru-mame will
remove all the trailing question marks (for example, 1998? becomes 1998) and
undetermined years will be expanded. For example, a game with a year
defined as 199? will be included in the filter
year == 1992
. This year expansion can be controlled with the option
<YearsOpts>YearExpansion</YearsOpts>
. Most user
will not want this option at all because games with unknown release years
are usually either fruit machines or low-quality bootlegs.
After you have configured your source and destination directories and
your filters, you can check what games will be copied into the
destination directory with the check-filter
command.
For the example, for the main filter configured above the results will be
something like this.
$ ./xru-mame.py check-filter main XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Checking filter] Filter name = main [Parsing MAME merged XML] Parsing MAME merged XML file ./DAT-mame/mame-0154-merged.xml... done Total number of games = 30551 Number of parents = 10038 Number of clones = 20513 [Reading ROMs in source directory] [Applying MAME filters] NOTE: -vv if you want to see filters in action <Main filter> Default filter, removing devices - Removed = 907 / Remaining = 29644 Filtering out clones - Removed = 20513 / Remaining = 9131 User wants games with samples Filtering out mechanical games - Removed = 2622 / Remaining = 6509 Filtering out BIOS - Removed = 67 / Remaining = 6442 Filtering out Non-Working games - Removed = 2627 / Remaining = 3815 <Driver filter> Filter = "cps1 or cps2" Removed = 3742 / Remaining = 73 <Categories filter> Filter = "not Mature and not Casino" Removed = 0 / Remaining = 73 <Controls filter> Filter = "Joy or Lightgun and not Mahjong and not Hanafuda" Removed = 4 / Remaining = 69 <Buttons filter> Filter = "buttons < 8" Removed = 0 / Remaining = 69 <Players filter> Filter = "players < 8" Removed = 0 / Remaining = 69 <Year filter> Filter = "year >= 1985 and year < 2005" Year expansion deactivated Removed = 0 / Remaining = 69 [Adding ROM dependencies] Game mpang depends on device qsound - Adding to list [Filtered game list] <Game> 1941 - Have ROM - 1941: Counter Attack (World 900227) <Game> 1944 - Have ROM - 1944: The Loop Master (USA 000620) ... <Game> xmvsf - Have ROM - X-Men Vs. Street Fighter (Euro 961004) [Report] Number of filtered ROMs = 70 Number of have ROMs = 70 Number of missing ROMs = 0
Note than in this list only the filtered games are printed. You can see if you have that ROM or not (missing ROM) in the source directory.
Once you are happy with your filter then it's time to copy your ROMs to the destination directory.
The copy <filterName>
command will filter
your ROMs and then copy, one by one, every filtered ROMs into the
destination directory.
$ ./xru-mame.py copy main XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Copy/Update ROMs] Filter name = main [Parsing MAME merged XML] Parsing MAME merged XML file ./DAT-mame/mame-0154-merged.xml... done Total number of games = 30551 Number of parents = 10038 Number of clones = 20513 [Reading ROMs in source directory] [Applying MAME filters] NOTE: -vv if you want to see filters in action <Main filter> Default filter, removing devices - Removed = 907 / Remaining = 29644 Filtering out clones - Removed = 20513 / Remaining = 9131 User wants games with samples Filtering out mechanical games - Removed = 2622 / Remaining = 6509 Filtering out BIOS - Removed = 67 / Remaining = 6442 Filtering out Non-Working games - Removed = 2627 / Remaining = 3815 <Driver filter> Filter = "cps1 or cps2" Removed = 3742 / Remaining = 73 <Categories filter> Filter = "not Mature and not Casino" Removed = 0 / Remaining = 73 <Controls filter> Filter = "Joy or Lightgun and not Mahjong and not Hanafuda" Removed = 4 / Remaining = 69 <Buttons filter> Filter = "buttons < 8" Removed = 0 / Remaining = 69 <Players filter> Filter = "players < 8" Removed = 0 / Remaining = 69 <Year filter> Filter = "year >= 1985 and year < 2005" Year expansion deactivated Removed = 0 / Remaining = 69 [Adding ROM dependencies] Game mpang depends on device qsound - Adding to list [Creating list of ROMs to be copied/updated] Added 70 ROMs [Copying ROMs into destDir] 0% <Copied> 1941.zip 1% <Copied> 1944.zip ... 98% <Copied> xmvsf.zip [Report] Copied ROMs 70
The update <filterName>
command operates
in a similar way to copy
. However, rather than
copying all the ROMs, each ROM in sourceDir is compared with the same
named ROM in destDir, and the file will be copied only if the destination
ROM does not exist or the file sizes are different. This is very
useful when you destination directory is network-mounted.
By default, only the newly copied files are displayed during the update
process. If you want to check also the updated files, then use the
option -v
in the command line.
With both copy
and update
commands, you can use the flags --dryRun
and
--cleanROMs
.
--dryRun
will not perform any operation in your
files. Use this flag if you want to test what will happen with your ROMs,
and if you are happy with the results execute
xru-mame without --dryRun
option. Note that if you use --dryRun
,
--cleanROMs
will
show you inaccurate outcomes, since the destination directory is not modified.
--cleanROMs
will delete any unknown ROM in the
destination directory. After copying/update the filtered ROMs, the
destination directory will be scanned and any ROM not found in the list of
filtered ROMs will be deleted. Use this with caution! The list of deleted ROMs
in destDir is always printed.
Both Advanced Launcher and ROM Collection Browser create NFO files having the scrapped ROM information, like year of the release, genre, etc. A typical NFO file looks like this.
<?xml version="1.0" ?> <game> <title>Toki (World, set 1)</title> <platform>MAME</platform> <year>1989</year> <publisher>TAD Corporation</publisher> <genre>Platform</genre> <plot/> </game>
The scrapping process may be extremely slow if your ROM collection is
huge. In order to speed up your
MAME ROM collection scanning, you can create NFO files if you specify
the --generateNFO
when invoking the copy
or
update
commands and then use the offline scrapers.
Note that the generated files will have and empty plot (maybe in a future version
of xru-mame the game plot will be extracted
from history.dat
).
Use the --cleanNFO
to remove any unknown
NFO files, in a similar way to the --cleanROMs
flag. The
list of deleted NFO files in destDir is always printed.
Regarding Artwork files, have a look first at the Artwork section of xru-console because in xru-mame this works in a very similar way. The main difference is that complete collections of Artwork for MAME exist, so there is no need for artwork substitution in xru-mame. Either you have the artwork or not.
If you have local Artwork available and want to copy it into destination folders (synchronised with your filtered ROM list), then configure the following into your MAME ROM filter.
<MAMEFilter> ... <ThumbsSource>/home/xbmc/roms/ArtWork/mame-titles/</ThumbsSource> <FanartSource>/home/xbmc/roms/ArtWork/mame-snaps/</FanartSource> <ThumbsDest >/home/NUC-remote/ROMs/ArtWork/mame-thumbs/</ThumbsDest> <FanartDest >/home/NUC-remote/ROMs/ArtWork/mame-fanart/</FanartDest> ... </MAMEFilter>
You may check you configuration with the
check-artwork <filterName>
command. For example:
$ ./xru-console.py check-artwork main XBMC ROM utilities - MAME edition version 0.1.0 [Parsing config file] [Check-ArtWork] Filter name = main [Parsing MAME merged XML] Parsing MAME merged XML file ./DAT-mame/mame-0154-merged.xml... done Total number of games = 30551 Number of parents = 10038 Number of clones = 20513 [Applying MAME filters] NOTE: -vv if you want to see filters in action <Main filter> Default filter, removing devices - Removed = 907 / Remaining = 29644 Filtering out clones - Removed = 20513 / Remaining = 9131 User wants games with samples Filtering out mechanical games - Removed = 2622 / Remaining = 6509 Filtering out BIOS - Removed = 67 / Remaining = 6442 Filtering out Non-Working games - Removed = 2627 / Remaining = 3815 <Driver filter> Filter = "cps1 or cps2" Removed = 3742 / Remaining = 73 <Categories filter> Filter = "not Mature and not Casino" Removed = 0 / Remaining = 73 <Controls filter> Filter = "Joy or Lightgun and not Mahjong and not Hanafuda" Removed = 4 / Remaining = 69 <Buttons filter> Filter = "buttons < 8" Removed = 0 / Remaining = 69 <Players filter> Filter = "players < 8" Removed = 0 / Remaining = 69 <Year filter> Filter = "year >= 1985 and year < 2005" Year expansion deactivated Removed = 0 / Remaining = 69 [Adding ROM dependencies] Game mpang depends on device qsound - Adding to list [Creating list of ROMs to be copied/updated] Added 70 ROMs [Artwork report] << ROM >> 1941.zip Original 1941 Have T 1941.png Have F 1941.png << ROM >> 1944.zip Original 1944 Have T 1944.png Have F 1944.png ... << ROM >> xmvsf.zip Original xmvsf Have T xmvsf.png Have F xmvsf.png Number of ROMs in destDir = 70 Number of ArtWork found = 70 Number of original ArtWork = 70 Number of replaced ArtWork = 0 Number of have Thumbs = 70 Number of missing Thumbs = 0 Number of have Fanart = 70 Number of missing Fanart = 0
Note that in xru-mame no artwork substitution is done. All artwork should be either original or missing and if not that is a bug.
When you are happy with the results, you can copy/update your
artwork with the commands copy-artwork <filterName>
and update-artwork <filterName>
. You can use the
--cleanArtWork
flag if you want to delete
unknown artwork into the thumbs and fanart destination directories.
xru-launcher-AL is a small utility that reads XBMC's Advanced Launcher configuration file and checks the launchers configured by the user. It reports if there are missing/unknown ROMs in your launcher and tells what launchers need to be updated.
$ xru-launcher-AL.py [-options] command
The following commands are supported.
usage
Print usage information.
list
Lists every launcher found in Advanced Launcher configuration file.
list-config
Lists every launcher found in the configuration file.
check
Checks the Advanced Launcher configuration file and compares against the ROM directories. It reports if Advanced Launcher should rescan the ROM launchers if outdated.
The following options are supported.
-h, --help
Print short command reference.
-v, --verbose
Print more information about what is going on.
-l, --log
Save program output in xru-launcher-AL-log.txt
xru-launcher-AL expects to be run on a different computer where your XBMC media centre is. The XBMC user's directory is network-mounted in the computer you run xru-launcher-AL using FUSE sshfs, nfs, samba, etc. However, it can also be run on the HTPC computer.
To start using xru-launcher-AL, create a
minimalistic configuration file named
xru-launcher-AL-config.xml
.
<AdvancedLauncher> <configFile>/home/NUC-remote/.xbmc/userdata/addon_data/plugin.program.advanced.launcher/launchers.xml</configFile> </AdvancedLauncher>
The tag configFile
configures the location of
Advanced Launcher's
configuration file. Next, use the list
command to read
Advanced Launcher's
configuration file and list every launcher found there. You should get
something like this.
$ ./xru-launcher-AL.py list XBMC ROM utilities - Advanced Launcher version 0.1.0 [Parsing config file] [Listing Advanced Launcher launchers] Parsing Advanced Launcher configuration file... done <Launcher> lauch_name = "Game Boy Advance (RetroArch VBA-Next)" lauch_application = /home/xbmc/bin/bin-retroarch/retroarch lauch_args = -L /home/xbmc/bin/bin-libretro/vba_next_libretro.so "%rom%" lauch_rompath = /home/xbmc/ROMs/roms-nintendo-gbadvance-nointro/ lauch_thumbpath = /home/xbmc/ROMs/artwork-AL/artwork-nintendo-gbadvance/thumbs/ lauch_fanartpath = /home/xbmc/ROMs/artwork-AL/artwork-nintendo-gbadvance/fanart/ lauch_romext = zip ...
For every launcher you want to check you need to create an entry
in xru-launcher-AL configuration file. To
check the launcher in the example, add this to the configuration
file xru-launcher-AL-config.xml
.
<AdvancedLauncher> <configFile>/home/NUC-remote/.xbmc/userdata/addon_data/plugin.program.advanced.launcher/launchers.xml</configFile> <launcher name="Game Boy Advance (RetroArch VBA-Next)"> <ROMsDest >/home/NUC-remote/ROMs/roms-nintendo-gbadvance-nointro/</ROMsDest> <FanartDest>/home/NUC-remote/ROMs/artwork-AL/artwork-nintendo-gbadvance/fanart/</FanartDest> <ThumbsDest>/home/NUC-remote/ROMs/artwork-AL/artwork-nintendo-gbadvance/thumbs/</ThumbsDest> </launcher> </AdvancedLauncher>
Because xru-launcher-AL runs on a different computer that Advanced Launcher, directory names will be different, so you have to tell xru-launcher-AL the directory location on the local computer of the remote (network mounted) directories. Only the launchers listed in xru-launcher-AL's configuration file will be checked.
Once you have finished editing the configuration file, you can
check the configured launchers with the list-config
command.
For example.
$ ./xru-launcher-AL.py list-config XBMC ROM utilities - Advanced Launcher version 0.1.0 [Parsing config file] [Listing configuration] Advanced Launcher configuration file /home/NUC-remote/.xbmc/userdata/addon_data/plugin.program.advanced.launcher/launchers.xml <Launcher> name = Game Boy Advance (RetroArch VBA-Next) destDir = /home/NUC-remote/ROMs/roms-nintendo-gbadvance-nointro/ fanartDestDir = /home/NUC-remote/ROMs/artwork-AL/artwork-nintendo-gbadvance/fanart/ thumbsDestDir = /home/NUC-remote/ROMs/artwork-AL/artwork-nintendo-gbadvance/thumbs/
To check your launchers use the check
command. You will
see some information of the process and at the end a report of the launchers not
checked, launchers checked and up to date, and launchers checked which need
an update.
$ ./xru-launcher-AL.py check XBMC ROM utilities - Advanced Launcher version 0.1.0 [Parsing config file] [Checking Advanced Launcher launchers] Parsing Advanced Launcher configuration file... done <Launcher> 'Game Boy Advance (RetroArch VBA-Next)' Not found AL configuration file ROM path Using configured ROM local path for this launcher 0 not found ROMs in AL configuration file 0 missing ROMs in AL configuration file ... <Launcher> 'SNES (Mednafen)' Not found AL configuration file ROM path Using configured ROM local path for this launcher 0 not found ROMs in AL configuration file 0 missing ROMs in AL configuration file [Report] The following launchers were not checked Playstation (Mednafen) The following launchers were checked and are up to date MAME MAME CPS1 MAME CPS2 MAME CPS3 MAME Neo-Geo SNES (Mednafen) Advanced Launcher needs an update for the following launchers Game Boy Color (Mednafen) Game Gear (Mednafen) Mega Drive 32X (RetroArch Picodrive)
You have a No-Intro ROM collection of the Sega Genesis and want to filter this collection to remove clone ROMs and Japanese ROMs. This is the situation.
The ROM collection is stored in your desktop computer in
the directory /home/wintermute/No-Intro/Sega Genesis/
.
XBMC is installed on a dedicated HTPC, which uses XBMCbuntu. XBMC runs with user xbmc.
The xbmc user's home directory is network-mounted in the
desktop computer using FUSE's sshfs in
/home/XBMC-remote/
.
You want to store your ROM collection in the HTPC computer in
the directory /home/xbmc/ROMs/roms-sega-genesis/
. Note
that in the desktop computer this directory is
/home/XBMC-remote/ROMs/roms-sega-genesis/
.
You prefer European/USA ROMs over Japanese ones. Also, you want to remove all Japanese games.
First, create the destination directories. Make sure your HTPC is network-mounted.
$ mkdir /home/XBMC-remote/ROMs/roms-sega-genesis
Then, create the configuration file
xru-console-config.xml
.
<ROMcollections> <collection name="Sega Mega Drive" shortname="genesis"> <ROMsSource>/home/wintermute/No-Intro/Sega Genesis/</ROMsSource> <ROMsDest >/home/XBMC-remote/ROMs/roms-sega-genesis/</ROMsDest> <filterUpTags>Europe, World, Rev2, Rev 1, Rev A</filterUpTags> <filterDownTags>Japan, Beta, Proto</filterDownTags> <includeTags>Europe, USA</includeTags> <excludeTags>Japan</excludeTags> </collection> </ROMcollections>
Next, check everything is all-right.
$ ./xru-console.py check-filter genesis
Finally, copy the ROMs.
$ ./xru-console.py update genesis
You have a MAME ROM collection
and want to split it into 3 folders: First one will have most
games, second folder will have only CPS1 games, and third folder
will have only CPS2 games. The ROM collection is stored in your desktop
computer in the directory /home/wintermute/MAME/roms/
.
The xbmc user's home directory is network-mounted in the
desktop computer using FUSE's sshfs in
/home/XBMC-remote/
.
First, create the destination directories. Make sure your HTPC is network-mounted.
$ pwd /home/wintermute $ mkdir DAT-mame $ mkdir /home/XBMC-remote/ROMs/roms-mame-main $ mkdir /home/XBMC-remote/ROMs/roms-mame-cps1 $ mkdir /home/XBMC-remote/ROMs/roms-mame-cps2
Create the MAME game database as described here.
Then, create the configuration file
xru-mame-config.xml
.
<MAMEConfig> <General> <MAME_XML >./DAT-mame/mame-0154.xml</MAME_XML> <MAME_XML_redux>./DAT-mame/mame-0154-reduced.xml</MAME_XML_redux> <Catver >./DAT-mame/Catver.ini</Catver> <MergedInfo >./DAT-mame/mame-0154-reduced.xml</MergedInfo> </General> <MAMEFilter name="main"> <ROMsSource>/home/wintermute/MAME/roms-mame/</ROMsSource> <ROMsDest >/home/NUC-remote/ROMs/roms-mame-main/</ROMsDest> <MainFilter>NoClones, NoMechanical, NoBIOS, NoNonworking</MainFilter> <Driver>not cps1 and not cps2 and not cps3 and not neogeo_noslot</Driver> <Categories>not Casino and not PinMAME and not Fruit_Machines</Categories> <Controls>Joy and not(Mahjong or Gambling or Hanafuda)</Controls> <Buttons>buttons <= 8</Buttons> <Players></Players> <Years></Years> </MAMEFilter> <MAMEFilter name="cps1"> <ROMsSource>/home/wintermute/MAME/roms-mame/</ROMsSource> <ROMsDest >/home/NUC-remote/ROMs/roms-mame-cps1/</ROMsDest> <MainFilter>NoClones, NoSamples, NoMechanical</MainFilter> <Driver>cps1</Driver> </MAMEFilter> <MAMEFilter name="cps2"> <ROMsSource>/home/wintermute/MAME/roms-mame/</ROMsSource> <ROMsDest >/home/NUC-remote/ROMs/roms-mame-cps2/</ROMsDest> <MainFilter>NoClones, NoSamples, NoMechanical</MainFilter> <Driver>cps2</Driver> </MAMEFilter> </MAMEConfig>
Next, check everything is all-right.
$ ./xru-mame.py check-filter main $ ./xru-mame.py check-filter cps1 $ ./xru-mame.py check-filter cps2
Finally, copy the ROMs.
$ ./xru-mame.py update main $ ./xru-mame.py update cps1 $ ./xru-mame.py update cps2