Dune HD

From YobiWiki
Jump to navigation Jump to search

Dune HD Max

Current firmware: 111122_0159_beta

Official Links:

Forums:

News:

Misc Links:

Tuning

DSF

See http://scottjohnson.org/wiki/Dune/Hacking for:

  • packing/unpacking DSF
  • compile your own software
  • make your own binary DSF

See here for misc utilities

optware

Why not being able to do sth like apt-get on the Dune to install many packages? (not tried yet)

Telnet

telnetd binary is already in the firmware, you just need to activate it, e.g. by using dune_service_telnetd.dsf

To start always telnetd at startup: 

touch /config/telnetd

You should see now on bottom right of the screen /setup/information: DEBUG MODE

IP control

No UPnP controllable, but much better IMHO, a specific HTTP protocol easy to implement and very complete.
More info on media_url features.

See below for an Android application implementing it for a remote control.
Here is an HTML file implementation for controlling from any browser.
here is a Chrome extension doing the same.
Here is a pretty complete application for Windows.

dune_folder.txt

This is a mechanism allowing easy UI customization.
Third party softwares exporting to that format:

AAI image format

Official description: AAImageGen-README.txt and tool AAImageGen.exe
It works well with Wine and can even be called in command line: 

wine AAImageGen.exe test.aai # and it'll generate a test.png
wine AAImageGen.exe test.png # and it'll generate a test.aai


Alternate project: aaimageviewer with read/write support, not working under Wine.

Quick Python script to convert AAI images to any other format 

#!/usr/bin/env python

import os, sys
import Image
for infile in sys.argv[1:]:
    f, e = os.path.splitext(infile)
    if e != ".aai":
        raise IOError, "Sorry I only accept .aai files"
    try:
        data=open(infile).read()
        size = int(data[:4][::-1].encode('hex'),16), int(data[4:8][::-1].encode('hex'),16)
        # No BGRA so we'll take it as RGBA then swap channels
        im = Image.fromstring("RGBA", size, data[8:], "raw", "RGBA")
        r, g, b, a = im.split()
        im = Image.merge("RGBA", (b, g, r, a))
        # Just change the extension to save under another format
        im.save(f + ".jpg", quality=95)
    except IOError:
        print "cannot convert", infile

Renaming harddrive

By default the internal HDD gets a name based on a UUID à la DuneHDD_6db00183_3f29_4474_ae8a_6b27fbf24304, which is not easy to handle when you want to access it via e.g. Samba.
Drop a dune_folder.txt file on the root of the HDD with as content the shortcut name you want, e.g.: 

storage_name = DuneHDD

Then reboot, the shared drive will now be seen as DuneHDD

Internal storage

Some functions and BD Live require an internal storage. This can be defined on a specific partition of the HDD but it's easier to keep a separate disk for that and be able to swap HDDs without worrying about the internal storage. The Dune HD Max has an extra USB port on the main board under the HDD rack metallic plate so we can plug a small 2Gb flash drive there and forget it.
Make sure it's empty otherwise the system will refuse to use it for internal storage (you can still erase everything from the UI then try to reconfigure it as internal storage)
Internal storage is mounted as /permanentfs

Video

See http://hdlandforum.fr/viewtopic.php?f=25&t=1049
My TV supporting all modes, I've setup mode 1080p 50Hz and auto framerate 24/50/60Hz

FlashLite apps

Streaming TV

To test streams, one can use the IP control protocol: 

wget -O - "http://dune/cgi-bin/do?cmd=start_file_playback&media_url=http://stream02.gtk.hu/bptv"
wget -O - "http://dune/cgi-bin/do?cmd=start_file_playback&media_url=http://80.249.172.28/autonomia"
wget -O - "http://dune/cgi-bin/do?cmd=start_file_playback&media_url=mms://vipmms9.yacast.net/bfm_bfmtv"

One can create a bookmark such as this one to tell the Dune HD device to play the current URL: 

javascript:document.location='http://dune/cgi-bin/do?cmd=start_file_playback&media_url='+encodeURIComponent(location.href)

If you use NoScript Application Boundaries Enforcer, you need to add an exception before "Site LOCAL" in the ABE preferences 

Site dune
Accept ALL
Deny


This will work only if the URL is the direct link to the video file.
If you want it running with e.g. Youtube you need first some URL extractor.
One way is to use a Firefox extension such as Unplug.
So you can use first Unplug to get the media URL then use this javascript bookmark.
Hopefully Unplug is opensource so we can get the sources and patch them to add directly a "Open in DuneHD" method: 

diff --git a/unplug@compunach/chrome/content/display/download.js b/unplug@compunach/chrome/content/display/download.js
index 6454d30..af644ab 100644
--- a/unplug@compunach/chrome/content/display/download.js
+++ b/unplug@compunach/chrome/content/display/download.js
@@ -580,6 +580,18 @@ UnPlug2DownloadMethods.add_button("open-tab", {
        group : "open"
 });
 
+UnPlug2DownloadMethods.add_button("open-DuneHD", {
+       avail : (function (res) {
+               return (res.download.url ? true : false);
+       }),
+       exec  : (function (res) {
+               UnPlug2SearchPage._win.location = "http://dune/cgi-bin/do?cmd=start_file_playback&media_url="+encodeURIComponent(res.download.url);
+       }),
+       obscurity : 100,
+       css : "open open-over",
+       group : "open"
+});
+
 UnPlug2DownloadMethods.add_button("open-new", {
        avail : (function (res) {
                return (res.download.url ? true : false);
diff --git a/unplug@compunach/chrome/locale/en-US/strings.txt b/unplug@compunach/chrome/locale/en-US/strings.txt
index 6356ced..b72e0a4 100644
--- a/unplug@compunach/chrome/locale/en-US/strings.txt
+++ b/unplug@compunach/chrome/locale/en-US/strings.txt
@@ -32,6 +32,9 @@ save_to_directory=Save to Folder
 dmethod.open-tab=Open in a new tab
 dmethod.open-tab.a=t
 dmethod.open-tab.tip=Open in a new tab.
+dmethod.open-DuneHD=Open in Dune HD
+dmethod.open-DuneHD.a=d
+dmethod.open-DuneHD.tip=Open in Dune HD.
 dmethod.open-new=Open in a new window
 dmethod.open-new.a=n
 dmethod.open-new.tip=Open in a new window.

IPTV

To test streams, one can use the IP control protocol: 

wget -O - "http://dune/cgi-bin/do?cmd=start_file_playback&media_url=udp://@239.255.1.138:2276"

I get the BelgacomTV IPTV via a separate network so the idea is to access it via eth0 while the regular network remains the Wi-Fi one.
Once the Wi-Fi connection is active, we ca do the following to be able to use the multicast IPTV network on eth0: 

route del -net 224.0.0.0 netmask 240.0.0.0 2>/dev/null
route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0 2>/dev/null

Streaming radio

Custom GUI skins

GUI skins can be switched via the setting "Setup / Appearance / Skin", or via "B" RC button on the main screen (and also via "ZOOM" RC button on any screen).

Plugins

Multi-region

I didn't try yet but by playing DVD iso files I got the following experience: a first message telling me that the region was not ok, a kind of mini-reboot and then the DVD file played properly. To be investigated.
dune_service_region_switch.dsf, to change: press 4 times on "Mute" then quickly on 1, 2 or 3 for region A, B or C

Zappiti

Done by HD Land
There is an ipad version too, for Android see below for a compatible app.
The only bemol is that it's a Windows-only program (as all the other alternatives I think) so I'm using it in a Virtualbox with Windows 7.
I had issues with v2.3.1094.0 so I'm currently using v2.1.2735.100
Note that since v2.3.1097 HD Land is restricting its program for French people who didn't buy their Dune HD via HD Land. For others there is a possibility to get licensing key if they prove they live and bought their product out of France. Maybe a good time to start looking at yaDIS...

Config

Some notes on my config (just for myself to remember) 

Dossier de configuration: F:\@zappiti\@db  (F: virtualbox shared /media/MyBook2/@dune)
Dossier de partage: F:\@DuneHDD-mirror ou Z:\   (Z: \\dune\DuneHDD)
Nouveau dossier films:
  F:\@DuneHDD-mirror\films               
  Réécrire en: storage_label://DuneHDD/films

Zappiti works on a local copy of my movies collection and links are rewritten when exported to the Dune HD as the movies are also present locally on its HDD

Exported index

The exported index (_Zappiti/ and Zappiti_Resources/) has then to be copied on the Dune HD and the Android tablet for a faster rendering
Options:

Probably the best option. Very easy to install (cf link above) and to use. It's a patched version that creates modules for each mounted resource
To get modules names: 

$ rsync dune::
DuneHDD        	Dune storage named DuneHDD

To push Zappiti data: 

$ rsync -av --delete ...DuneHDD-mirror/_Zappiti          dune::DuneHDD
$ rsync -av --delete ...DuneHDD-mirror/Zappiti_Resources dune::DuneHDD

Note that if you try to copy Zappiti indexes from a locally attached drive via the Dune interface you can easily reach the maximum of 9999 files in _Zappiti. Better to initiate the copy via a telnet session.

Using rsync to push indexes to the Android tablet (rooted, using SSHDroidPro & rsync binary): 

$ rsync -av --delete ...DuneHDD-mirror/_Zappiti          root@tablet:/sdcard/Zappiti/
$ rsync -av --delete ...DuneHDD-mirror/Zappiti_Resources root@tablet:/sdcard/Zappiti/

Mangling database

If you want to move or rename files or folders, Zappiti wants to scrap them again and disregard the old entries.
To avoid that, you can edit directly the sqlite database.
Just a little problem: the database is encrypted by using ASP .NET connectionstrings (cf possible implementation [Using http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki here]).
So we'll follow [ http://www.mpcclub.com/forum/archive/index.php/t-27073.html this howto]:
Open 

C:\Program Files (x86)\HDLand\Zappiti\Zappiti.exe.config
<connectionStrings>
   <add name="Entities" connectionString="metadata=res://*/Models.Models.csdl|res://*/Models.Models.ssdl|res://*/Models.Models.msl;provider=System.Data.SQLite;provider connection string="data source=D:\Works\NETil\Works\Companies\HDLand\DuneMediaManager_b1\DuneMediaManager\Data\database.db;password=BLABLA"" providerName="System.Data.EntityClient" />
 </connectionStrings>

Here password is BLABLA.
Get Sqlite2009 Pro Enterprise Manager, again a Windows stuff but no much option to use this .NET encryption (sth with RSA but still using a symmetric password, no idea what this means) 

- Open database
- select RSA & provide password

Then you can make your SQL queries, e.g. 

UPDATE 'Media' set Path=REPLACE(Path, 'F:\', 'F:\@DuneHDD-mirror\');
UPDATE 'TVShowFolder' set Folder=REPLACE(Folder, 'F:\', 'F:\@DuneHDD-mirror\')

To export the DB in an unencrypted SQL dump, press F10.

Exporting pictures & IMDB IDs

After having discovered YAMJ, I decided to move away from Zappiti. Not that Zappiti is bad but having to run it under Windows 7 under VirtualBox and to access remotely all movies in and data out is a real pain.
In the previous paragraph you saw how to export the encrypted DB into plain SQL.
Here are very ugly notes (when job is to be done only once I tend to write ugly bash/sed/awk oneliners)
To recover matches between movie names and IMDB IDs: 

cat database-2.1.2735.100.sql |recode latin1..utf8|egrep '(<name>|<imdb_id>)' |sed 's/^.*<name>//;s/<\/name>//'|awk '/^[^[:space:]]/{title=$0}/<imdb_id>/{imdb=$0; print title imdb}'|sort|uniq > imdblist

To recover anonymized images from Pictures/ (which are all Jpeg BTW)
Get matches between movie internal ID and fanart ID and cover ID: 

 egrep -i "insert into groupe values" database-2.1.2735.100.sql|recode latin1..utf8 > database-2.1.2735.100.groupe.sql
 awk -F, '{sub(/INSERT INTO Groupe VALUES\(/,"",$1); print $1, $4, $5}' database-2.1.2735.100.groupe.sql|sort > database-2.1.2735.100.groupe.small

Get matches between movie internal ID and movie filename (and transcode Windows path to local unix path) 

 egrep -i "insert into media values" database-2.1.2735.100.sql|recode latin1..utf8 > database-2.1.2735.100.media.sql
 awk -F, '{sub(/INSERT INTO Media VALUES\(/,"",$1);id=$1;sub(/.*(.)F:\\@DuneHDD-mirror/,"\"");sub(/. (NULL|[^A-Za-z0-9][A-Z][A-Z]).*/,"\"");gsub(/\\/,"/"); print id,$1}' database-2.1.2735.100.media.sql |sed "s/''/'/g"|sort > database-2.1.2735.100.media.small

Join those two based on movie internal ID 

 join database-2.1.2735.100.groupe.small database-2.1.2735.100.media.small > database-2.1.2735.100.small

Let's see on which entries it could break: 

 egrep -v ".(avi|AVI|Avi|mpg|MPG|Mpg|mkv|MKV|Mkv|iso|ISO|Iso|mp4|MP4)\"$" database-2.1.2735.100.small

Ok we can remove them 

 egrep ".(avi|AVI|Avi|mpg|MPG|Mpg|mkv|MKV|Mkv|iso|ISO|Iso|mp4|MP4)\"$" database-2.1.2735.100.small > database-2.1.2735.100.small2

Recover fanarts 

 grep -v "^[^[:space:]]\+ NULL" database-2.1.2735.100.small2|awk '{id=$2;gsub(/[^[:alnum:]]/,"",id); $1="";$2="";$3="";sub(/[[:space:]]*"/,"\".");sub(/.(avi|AVI|Avi|mpg|MPG|Mpg|mkv|MKV|Mkv|iso|ISO|Iso|mp4|MP4)\"$/,"");system("mkdir -p \"$(dirname "$0"\")\"; cp Pictures/" id " " $0 ".fanart.jpg\"")}'

Recover covers 

 grep -v "^[^[:space:]]\+ [^[:space:]]\+ NULL" database-2.1.2735.100.small2|awk '{id=$3;gsub(/[^[:alnum:]]/,"",id); $1="";$2="";$3="";sub(/[[:space:]]*"/,"\".");sub(/.(avi|AVI|Avi|mpg|MPG|Mpg|mkv|MKV|Mkv|iso|ISO|Iso|mp4|MP4)\"$/,"");system("mkdir -p \"$(dirname "$0"\")\"; cp Pictures/" id " " $0 ".jpg\"")}'

Find art not linked to a file (e.g. sets) 

 join -a 1 database-2.1.2735.100.groupe.small database-2.1.2735.100.media.small > database-2.1.2735.100.smallext
 cat database-2.1.2735.100.small database-2.1.2735.100.smallext |sort|uniq -u > database-2.1.2735.100.groupe.small.uniq
 mkdir others
 cat database-2.1.2735.100.groupe.small.uniq |awk '{gsub(/[^[:alnum:]]/,"",$2);gsub(/[^[:alnum:]]/,"",$3);if ($2!="NULL") print "cp Pictures/"$2" others/"$2".jpg";if ($3!="NULL") system("cp Pictures/"$3" others/"$3".jpg")}'

Some fanart were not flagged as such, find covers with wrong ratio: 

 find . -name "*.jpg" -exec jpeginfo -l {} \;|grep -v fanart|awk '{if ($1 > $3) print }'

yaDIS

Eversion and YAMJ

This is entirely different from the other approaches:

  • frontend / backend: Eversion is one of the compatible frontends, YAML is one of the compatible backends
  • frontend with flashlite
  • both are open-source
  • runs under Linux but not only, YAML is actually a Java CLI

On the dark side, things are not very well documented or to be more exact documentation is quite scattered...
Many combinations are possible but to get it running on Dune, apparently Eversion + YAMJ is the proper way to go.
My scrap notes, ongoing as everything is rather new to me...

Resources

Installation

Get both tools in a combined archive from Eversion download page. I took aio_eversion_yamj_r0179c.zip. See also Github downloads page.
Unzip
Before you run yamj the first time, edit the dune_folder.txt file found in yamj/skins/eversion/interface to adjust the path. the path will be swf://(dune compatible path )/Jukebox/eversion.swf 

apt-get install mediainfo

Edit moviejukebox.properties

  • uncomment mediainfo.home=/usr/bin
  • Add a link to the file where we'll describe sources: mjb.libraryRoot=library.xml
  • Add a link to the directory where we'll output the generated data: mjb.jukeboxRoot=/media/nas/Share/

Copy and adapt example-library.xml => library.xml, e.g. for a nas folder mounted locally and to be accessed by Dune over Samba: 

<path>/media/nas/Share/video/</path>
<playerpath>smb://user:pwd@nas_ip/Share/video/</playerpath>

Usage

Now we can run the tool: 

./MovieJukebox.sh

Warning when editing XML files: if you get the following error when running the tool "The reference to entity * must end with the ';' delimiter" it's because URI must be html-escaped, e.g. replace & by &

Quick way to test out the generated stuff: 

wget -q -O - "http://dune/cgi-bin/do?cmd=launch_media_url&media_url=swf://smb://user:pwd@nas_ip/Share/Jukebox/eversion.swf"

It works but it's pretty slow on DuneHD

YAMJ alone

Actually I almost never use the remote control to navigate the media library directly on the DuneHD.
I prefer to do it either from the PC or from the Android tablet.
We've seen eversion but it doesn't work on the PC neither on DMC for Android so it doesn't really fit my need...
YAMJ produces directly HTML which is ok for the PC as we can control the DuneHD by HTTP requests and it's also ok for the tablet via a web browser or via DMC as there is a script to convert YAMJ output to dune_folder structure as we'll see later.

Installation

I downloaded moviejukebox-2.8.zip from here
Some installation notes here

YAMJ Tuning

I'm using several setups for parents & kids, here e.g. I'll create a "foo" setup

library-foo.xml

Trick is to use Dune IP control in the URL
Example: 

<libraries>
    <library>
        <path>/media/nas/video/</path>
        <playerpath>http://ip_of_dune/cgi-bin/do?cmd=launch_media_url&media_url=smb://user:password@ip_of_nas/video/</playerpath>
        <exclude name="sample,tmp/,temp/,RECYCLE.BIN/,/._"/>
        <description>Movies</description>
        <prebuf></prebuf>
        <scrapeLibrary>true</scrapeLibrary>
    </library>
</libraries>

categories-foo.xml

It's a copy of categories-default.xml where I disable most categories, e.g. to keep it simple for my kid
Just change "<enable>false</enable>" into "<enable>false</enable>" for each category to suppress

moviejukebox-foo.properties

The main tuning file
Major stuffs: 

# This is the skin property, you will need to change this if you want to change the look of YAMJ
mjb.skin.dir=./skins/default
# new yamj feature to enabled directory hashing layout for image storage.  Results in more speed on larger jukeboxes
mjb.dirHash=true
# Unix/Linux/Mac users will need to install Mediainfo on their own and remove the # from the front of this setting
mediainfo.home=/usr/bin/
# Set to true to have YAMJ monitor your settings and adjust overwrites as needed automatically for you.
mjb.monitorJukeboxProperties=true
# Create TV show Boxsets automatically (group all tv show seasons together as 1 show)
mjb.singleSeriesPage=true
# threading for speed
mjb.MaxThreadsProcess=3
mjb.MaxThreadsDownload=3
mjb.MaxDownloadSlots=.*=2,.*imdb.*=3,.*google.*=3,.*yahoo.*=3,.*themoviedb.*=4,.*thetvdb.*=2,.*apple.*=1

Personal settings 

mjb.jukeboxRoot=/media/nas/Jukebox
mjb.libraryRoot=library-foo.xml
mjb.detailsDirName=Movies
mjb.xmlCategoryFile=categories-foo.xml

Reduce further indexes 

# Comma-separated list of indexes to generate. Valid indexes include:
#  Other, Genres, Title, Certification, Year, Library, Cast, Director, Writer and Country
mjb.categories.indexList=Other,Title,Library,Set

IMDB/TVDB languages 

imdb.site=fr
thetvdb.language=fr
#mjb.internet.plugin=com.moviejukebox.plugin.AllocinePlugin

MovieJukebox-foo.sh

Just MovieJukebox.sh with the proper options 

#!/bin/sh
java -Xms256m -Xmx1024m -classpath .:./resources:./lib/* com.moviejukebox.MovieJukebox -p moviejukebox-foo.properties -c "$@"

TODO

One broken thing is the "play all" button for series.
I'll have to convert the .jsp into proper .m3u playlists
Change in myserie.html href="myserie S01E01.playlist.jsp" into sth like

Force to fetch again default covers

We look for the default covers (well detection only based on filesize for speed reasons, use sha1sum if you want something more robust) in Jukebox and remove the corresponding movie XML file: 

find . -size 20524c|awk '{sub(/.jpg/,".xml");sub(/\/.\/..\//,"/"); system( "rm \""$0"\";\n")}'

Pydun

Converts a YAMJ structure into a dune_folder, useful for Android DMC
Quite rough
YAMJ libraries need to get a name (description)

Misc

  • Some notes on proper filenames for movie detection
  • Yayman is a GUI for YAMJ but it's not clear what it does, it already renamed all my versioned files in lib/ :-(

Video notes

To identify video files: Mediainfo, available in Debian as mediainfo & mediainfo-gui
MKV tools (mkvmerge, mkvinfo, mkvextract): mkvtoolnix & mkvtoolnix-gui in debian

I had issues to play some 1080p mkv. Remultiplexing them with a newer version of mkvmerge seems enough to fix the issue: 

mkvmerge -o new.mkv old.mkv

Dune HD has no problem handling latest mkvmerge versions with metadata compression, the problem seems to be specific to some older versions of mkvmerge.

Still, there is one type of video codec apparently not supported, flagged as "VfW compatible" (Video for Windows) by mkvmerge, mediainfo tells us it's actually codec V_REAL/RV40 == RealVideo 4.0 aka RealVideo 9

To merge 2 avi files (e.g. if the movie was splitted in 2 CDs), use avimerge: 

avimerge -o new.avi -i old-cd1.avi old-cd2.avi

Android

  • Dune Media Controller, supports the IP control protocol and the dune_folder.txt structures, a perfect complement to Zappiti.

Can work on remote index but much better if it has a local copy of the index (Zappiti indexes are huge)

Tests

Read tests

Internal HDD: 79MB/s
External HDD USB: 21MB/S (write to internal: 13MB/s)

HDNetwork

It is available only if the player license includes “HDNetwork” option; this can be checked in the player menu Setup > Information. If the “HDNetwork” option is purchased, all the basic player functionality remains available.
This looks like what could be found in conference centers, controlling many screens at once. IMHO you can already achieve the same with the IP control protocol. Probably it's sold as an expensive bundle for those specific customers.


Retrieved from "https://wiki.yobi.be/index.php?title=Dune_HD&oldid=7748"