Friday, May 1, 2009

Monitoring a Directory to Automatically Invoke HandBrake

I've seen this question pop up a lot on HandBrake's forums and IRC channel, so I thought I'd make an entry about it here (Mac users skip down to the bottom for your directions):

Many folks have expressed interest in being able to specify a directory for HandBrake to 'watch' for new files that it would then automatically attempt to convert with predefined settings. I think most people are wanting this for use with devices, such as iPods, PS3s and AppleTVs, which require specific settings for videos to work. While HB doesn't support this functionality on its own (and the devs don't sound too interested in adding it), you can accomplish much the same thing in Ubuntu Linux using HandBrakeCLI and a little shell scripting.
WARNING: I'm a novice at scripting and there is definitely a more effective and elegant way of doing this. If you have a suggestion, please leave a comment! Similar steps will also work on other platforms/distros, so feel free to leave a comment about your successes or failures.

First, we'll need to install a utility to enable monitoring of directories:
sudo aptitude install inotify-tools
Next, we'll make some new directories in our home folder to hold our scripts and conversions. In a terminal, type:
cd $HOME ; mkdir HandBrake ; mkdir HandBrake/convert
Navigate to the newly created HandBrake directory:
cd HandBrake
and type:
gedit monitor.sh
This is where we'll write our script to monitor the 'convert' directory and invoke another script to do the actual conversion:
#!/bin/bash

inotifywait --monitor -e moved_to -e create ~/HandBrake/convert | while read dir;
do
(~/HandBrake/convert.sh)

done
Save, exit and--again--type:
gedit convert.sh
Here we will create our conversion script (be sure to put your desired file extension and preset in place instead of the bracketed reminders):
#!/bin/bash
for file in ~/HandBrake/convert/*
do HandBrakeCLI -v -i "$file" -o "$file".converted.[FILE-EXTENSION-GOES-HERE] --preset [PRESET-NAME-GOES-HERE] ;

#uncomment next line to delete original
#rm $file

done
Save and exit, then type:
chmod +x *.sh
to make both scripts executable.

Now, you can start monitoring by typing:
sh ~/HandBrake/monitor.sh
or you can set the script to run as a startup item where it will run continuously in the background starting the next time you log on.

Henceforth, any file you move or copy into the 'convert' directory will automatically convert to the desired format. This script will only work on one file at a time (i.e., you have to wait for the encoding to finish before dropping in the next file to convert). Also of note: HB will choke if the file is weird in any way--e.g. no audio track--and you'll have no way of knowing it if the script is running in the background, since it won't print any output.

Good luck and let me know if you run into any problems.

Bulk Encoding on Macs


Update (06/01/09): There's been a lot of clamoring on the Mac board of the HandBrake forums asking for bulk input of files to be converted using a preset. The devs have no interest in adding such a feature at this time because of the tremendous support headache it could cause, but you Mac users can do scripting to accomplish the same thing.

Just like the Linux users, open a Terminal (Applications > Utilities > Terminal) and type:

cd Desktop ; nano convert.sh
then paste in this (shift+ctrl+v; be sure to put your desired file extension and preset in place instead of the bracketed reminders):
#!/bin/bash
for file in ./*
do ./HandBrakeCLI -v -i "$file" -o "$file".converted.[FILE-EXTENSION-GOES-HERE] --preset [PRESET-NAME-GOES-HERE] ;

#uncomment next line to delete original
#rm $file

done
Save and exit (ctrl+x), then type:
chmod +x *.sh
to make the script executable. Now, just put the script into a folder with your HandBrakeCLI binary and you should be able to invoke the script (navigate to its directory in the Terminal by typing cd [space after cd] and then dragging your conversion folder onto the Terminal window and hit 'Enter,' then type ./convert.sh) and automatically convert any files within the directory using the chosen preset. I would recommend just keeping a folder around that you use for conversions and keep the script and HandBrakeCLI binary in there at all times, then you can just drop in the files you want to convert, start the script and go along your merry way.

UPDATE (12/20/2013): An anonymous reader shared his script, which sounds much more robust than mine:
inotifywait -r --monitor --quiet -e moved_to -e close_write --format '%w%f' /mnt/public/convert/video/android-hq/ | while read -r FILE; do echo "File copy detected" echo "Starting HandBrake..." (sleep 15 && /usr/bin/HandBrakeCLI -v -i "$FILE" -o /mnt/public/convert/output/video/"$(echo "$FILE" | cut -c38- | rev | cut -c4-| rev)android-hq.mkv" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --width 720 --loose-crop --decomb --markers --turbo --two-pass --vfr --subtitle 1 --native-language eng && sleep 15 && rm -rf "$FILE")& done
UPDATE (9/27/2014): The same user (name is teeedubb, apparently) is back with an update. This was posted in the comments but got mangled, so I tried to piece it back together:
I'm back - I have revisited the above script because it had some glaring faults - it would start a handbrakecli process for each file, which when encoding a seasons worth of tv shows it would bring my pc to its knees, it didnt handle files in sub directories, didnt handle multiple profiles and probably would have given undesirable results if a file extension had more than 3 character, plus it was kinda messy. New version below solves these issues: It queues encodes using task-spooler (tsp package in ubuntu), works with sub-directories in the watch folder (and deletes any empty subdirectories within watch folders) and supports multiple profiles. Options are 'watch directory', 'output directory' and 'task-spooler slots' (concurrent jobs) which are set through the variables. You can set multiple handbrake profiles which the script uses based on which directory the files are copied into, if files are copied into the root watch directory the first profile is used. Options for profiles are: 'name', profile name which will be appended to the transcoded file (this needs to match the profile directory and be fairly unique - script searches the input file location for profile name, so a profile called 'android' could confuse the script when transcoding the movie 'android cop'), 'handbrake settings', settings for handbrake to use (I'm pretty sure you could use --preset XXXX here as well), 'file extension', file extension for handbrake to use on output file and 'delete source file', whether or not to delete the source video file. You can create extra profiles by adding variables beginning with PRESET2 etc and adding elif entries to the if/then statement in the script. By default the script attempts to delete any empty subdirectories *and its ancestors* when completing a transcode, so you need to ensure you profile directories are not empty - I have done this by creating a hidden file in each profile dir and making the file un-deletable (eg: profile1 dir is /mnt/public/convert/video/android-hq, run the command touch /mnt/public/convert/video/android-hq/.android-hq && sudo chattr +i /mnt/public/convert/video/android-hq/.android-hq).
and here's the script:
#!/bin/bash
#script to watch a directory for incoming files and pass them onto HandBrakeCLI

WATCH_DIR="/mnt/public/convert/video/"
OUTPUT_DIR="/mnt/public/convert/output/video/"
TASK_SPOOLER_SLOTS=2

##HANDBRAKE PROFILE 1
PRESET1_NAME="android-hq"
PRESET1_HANDBRAKE_SETTINGS=" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --width 720 --loose-crop --decomb --markers --turbo --two-pass --vfr --subtitle 1 --native-language eng"
PRESET1_FILE_TYPE="mkv"
PRESET1_DELETE_SOURCE="yes"

###########################

ts -S $TASK_SPOOLER_SLOTS
inotifywait --recursive --monitor --quiet -e moved_to -e close_write --format '%w%f' "$WATCH_DIR" | while read -r INPUT_FILE; do

PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"

if [[ $(echo "$WATCH_DIR" | grep -i "$PRESET1_NAME") ]] ; then
PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"
fi
FULL_FILE_NAME=$(echo ${INPUT_FILE##*/})
OUTPUT_FILE=$(echo ${FULL_FILE_NAME%.*})
tsp bash -c 'nice -n 19 /usr/bin/HandBrakeCLI -v -i "$0" -o "$1""$2"-"$3"."$4" "$5" && if [[ "$6" = yes ]] ; then sleep 15 ; rm -f "$0" ; fi ; rmdir -p "$(dirname "$0")"' "$INPUT_FILE" "$OUTPUT_DIR" "$OUTPUT_FILE" "$PRESET_NAME" "$FILE_TYPE" "$HANDBRAKE_SETTINGS" "$DELETE_SOURCE"
done

15 comments:

Ron said...

Thanks for the insight on bulk converting with handbrake. I never knew about inotifywait.

An issue I have run across with the scripts is handling of transfers to the folder. For example, if I rsync or scp a file to the convert directory, inotifywait will see the event and trigger a convert, but the file hasn't fully arrived yet.

Also, when the file does fully arrive, then another inotifywait event is triggered.

Hunter K. said...

Hi Ron,
Yeah, that's definitely an issue. If you don't need immediate conversion (i.e., you can wait a few hours for it to get started), I would probably recommend that you set up a cron job to run at, say, midnight, and have it invoke your script to convert any files in your desired directory. In this case, you would want to make sure that HandBrake outputs to a different directory (so the videos don't get re-converted next time the script runs) and that you have the script delete the originals (so they don't get converted over and over).

I'll be happy to help you with any of that if you need it.

Bee Mo said...

Thanks so much for this. I was searching for hours for a script that works. I had to change "./*" to the source directory to get this to work. But it's working beautifully.

Thanks for taking the time to post this.

Hunter K. said...

Hi Bee Mo,
I'm glad to hear it's working for you! :D

Anonymous said...

The script seems to start a Handbrake conversion immediately if a file is copied to the folder, right?
This means, if you copy ten files, you will have ten times handbrake running on your system, right?
Wouldn´t it make sense, to add new videos to encoding queue?

Anonymous said...

Hey man, thanks for this. Ive wanted something like this for a while but never got around to it and your post gave me a starting point. Ive made some changes which only start handbrake when the file has been finished copying and it can handle multiple files at at one by starting new handbrakecli processes - I dont know how well it will handle say 20 files at once but it works ok with 2-3 concurrent files. Also it doesnt seem to like folders, so I just place individual files in the folder.

inotifywait -r --monitor --quiet -e moved_to -e close_write --format '%w%f' /mnt/public/convert/video/android-hq/ | while read -r FILE;
do

echo "File copy detected"
echo "Starting HandBrake..."
(sleep 15 && /usr/bin/HandBrakeCLI -v -i "$FILE" -o /mnt/public/convert/output/video/"$(echo "$FILE" | cut -c38- | rev | cut -c4-| rev)android-hq.mkv" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --width 720 --loose-crop --decomb --markers --turbo --two-pass --vfr --subtitle 1 --native-language eng && sleep 15 && rm -rf "$FILE")&

done


Hunter K. said...

@Anon
Nice! Thanks for sharing :D

I added your script to the post.

teeedubb said...

Too many character for one post so here is the 2nd half:

Script file:
video-watch-converter.sh
#!/bin/bash
#script to watch a directory for incoming files and pass them onto HandBrakeCLI

WATCH_DIR="/mnt/public/convert/video/"
OUTPUT_DIR="/mnt/public/convert/output/video/"
TASK_SPOOLER_SLOTS=2

##HANDBRAKE PROFILE 1
PRESET1_NAME="android-hq"
PRESET1_HANDBRAKE_SETTINGS=" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --wi
PRESET1_FILE_TYPE="mkv"
PRESET1_DELETE_SOURCE="yes"

###########################

ts -S $TASK_SPOOLER_SLOTS
inotifywait --recursive --monitor --quiet -e moved_to -e close_write --format '%w%f' "$WATCH_DIR" | while read -r INPUT_FILE; do

PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"

if [[ $(echo "$WATCH_DIR" | grep -i "$PRESET1_NAME") ]] ; then
PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"
fi
FULL_FILE_NAME=$(echo ${INPUT_FILE##*/})
OUTPUT_FILE=$(echo ${FULL_FILE_NAME%.*})
tsp bash -c 'nice -n 19 /usr/bin/HandBrakeCLI -v -i "$0" -o "$1""$2"-"$3"."$4" "$5" && if [[ "$6" = yes ]] ; then sleep 1ETTINGS" "$DELETE_SOURCE"
done


A upstart script for ubuntu to make it easy to start/stop the script (use the command sudo initctl stop video_converter), you need to configure the script location and change 'prime' to the user you want the script run as:
/etc/init/video_converter.conf
# Auto Video Converter Script
#

description "Auto video converting"
author "teeedubb"

start on startup
stop on runlevel [!2345]
exec sudo -u prime bash /home/prime/.scripts/video-watch-converter.sh


Enjoy!

teeedubb said...

First half of my post has dissapeared? Took me a while to type that up :(
Also looks like the script isnt displayed properly, here is a pastebin link:
http://paste.ubuntu.com/8438645/

teeedubb said...

Glad you were able to salvage my first post, thanks. I noticed I made a mistake in the script - the line "ts -S $TASK_SPOOLER_SLOTS" should be "tsp -S $TASK_SPOOLER_SLOTS".

Karl said...

Hi, Thanks for the great, post, I see these threads are nearly a year old, so maybe out of date now?

I have tried to use the script that was posted in the comment that you added as an update, it worked for me the very first time, but whenever i try to run it now, it just sits there with a blinking cursor not doing anything...am I doing something wrong?

Thanks

Karl said...

Ignore my last comment, I have managed to get it working, the issue I have now is it is ignoring the settings I have defined in the "PRESET1_HANDBRAKE_SETTINGS=" and seems to be just encoding with some default settings? I can't see the console output to see what is going on...

Anonymous said...

hi, when I try run convert.sh script which teeedubb wrote.

I got the error.

24: convert.sh: [[: not found

Anonymous said...

+++

I got the error message but It works to convert.

Unknown said...

Is there a way to keep the directory structure from the input in the output? For example I currently have this happening:

/Input
/Directory1
File1.avi
/Directory2
File2.mov

/Output
File1.mkv
File2.mkv

I'm looking for this:

/Output
/Directory1
File1.mkv
/Directory2
File2.mkv

Analytics Tracking Footer