Webspiding
Revision as of 01:12, 2 March 2008 by <bdi>PhilippeTeuwen</bdi> (talk | contribs)
Converted with HTML::WikiConverter::MediaWiki from my old phpwiki site
Cartes satellite Ikonos
Status: Les coordonnées des 4 cartes de Liège ont été découvertes mais pas encore celles de Bruxelles
#!/bin/bash
###########################################################################
# Purpose of this script is to fetch parts of a map and reconstruct it
# It's suited to fecth maps available at www.geo.ulg.ac.be/edusat/en/ikonos
#
# Version 0.3 (08/12/2002) Copyright Doegox <philippe.teuwen@ael.be>
# This script is released under the GNU/General Public License
#
# History:
# * Version 0.3:
# XY Coord computation made easier, no more need for bc
# Support for additional maps: all Liege Ikonos are now supported
# * Version 0.2:
# Fix parametrization of total nr of images
# Fetch more robust: retry if download failed
###########################################################################
# Here are the parameters to adapt:
###########################################################################
# imagename=
## Give just a prefix for the saved images
# savedname=
## Size of the whole image to recover, quite hard to guess I agree...
# xbig=
# ybig=
## Size of the returned snapshots
xsnap=500
ysnap=500
###########################################################################
########## IKONOS Liege fusion
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cliege-ikonos-fusion.bmp"
# savedname="lg-fusion"
# xbig=18497
# ybig=11066
########## IKONOS Liege panchromatique
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cliege-ikonos-pan.bmp"
# savedname="lg-pan"
# xbig=11024
# ybig=11008
########## IKONOS Liege multispectral
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cliege-ikonos-rgb.bmp"
# savedname="lg-rgb"
# xbig=2756
# ybig=2752
########## IKONOS Liege infrared
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cliege-ikonos-nir.bmp"
# savedname="lg-nir"
# xbig=2756
# ybig=2752
########## IKONOS Bruxelles fusion
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cbruxelles-ikonos-fusion.bmp"
# savedname="bxl-fusion"
# xbig=
# ybig=
########## IKONOS Bruxelles panchromatique
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cbruxelles-ikonos-pan.bmp"
# savedname="bxl-pan"
# xbig=
# ybig=
########## IKONOS Bruxelles multispectral
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cbruxelles-ikonos-rgb.bmp"
# savedname="bxl-rgb"
# xbig=
# ybig=
########## IKONOS Bruxelles infrared
# imagename="d%3A%5CEdusat-Images%5Cikonos%5Cbruxelles-ikonos-nir.bmp"
# savedname="bxl-nir"
# xbig=
# ybig=
###########################################################################
###########################################################################
imax=$((xbig/xsnap+1))
jmax=$((ybig/ysnap+1))
for ((i=0;i<$imax;i++)); do
x=$(($i*$xsnap+$xsnap/2));
for ((j=0;j<$jmax;j++)); do
y=$(($j*$ysnap+$ysnap/2));
echo Getting bitmap $(($j+$i*$jmax+1))/$(($imax*$jmax)) @ x=$x y=$y
until wget --tries=inf --quiet "http://www.geo.ulg.ac.be/edusat/en/ikonos/cgi-bin/extr500.pl?nom=$imagename&quicklook-x-size=$xbig&quicklook-y-size=$ybig&image.x=$x&image.y=$y" --output-document=$savedname-$i-$j.bmp
do
sleep 1;
echo "Retry...";
done
echo -n "Converting to PNM... "
anytopnm $savedname-$i-$j.bmp > $savedname-$i-$j.pnm
rm $savedname-$i-$j.bmp
if [ $j != 0 ];
then
if [ $j != $(($jmax-1)) ];
then
echo "Merge 2 images in columns... "
pnmcat -topbottom $savedname-$i-$(($j-1))-merged.pnm $savedname-$i-$j.pnm > $savedname-$i-$j-merged.pnm
rm $savedname-$i-$(($j-1))-merged.pnm $savedname-$i-$j.pnm
else
echo "Cut last image of the column, there is some overlapping here... "
pnmcut -top $((-($ybig-$ybig/$ysnap*$ysnap))) $savedname-$i-$j.pnm > $savedname-$i-$j-cut.pnm
rm $savedname-$i-$j.pnm
echo "Finish merging this column... "
pnmcat -topbottom $savedname-$i-$(($j-1))-merged.pnm $savedname-$i-$j-cut.pnm > $savedname-$i-$j-merged.pnm
rm $savedname-$i-$(($j-1))-merged.pnm $savedname-$i-$j-cut.pnm
fi
else
mv $savedname-$i-$j.pnm $savedname-$i-$j-merged.pnm
fi
done
mv $savedname-$i-$(($jmax-1))-merged.pnm $savedname-$i.pnm
if [ $i != 0 ];
then
if [ $i != $(($imax-1)) ];
then
echo "Merge 2 columns... "
pnmcat -leftright $savedname-$(($i-1))-merged.pnm $savedname-$i.pnm > $savedname-$i-merged.pnm
rm $savedname-$(($i-1))-merged.pnm $savedname-$i.pnm
else
echo "Cut last column, there is some overlapping here... "
pnmcut -left $((-($xbig-$xbig/$xsnap*$xsnap))) $savedname-$i.pnm > $savedname-$i-cut.pnm
rm $savedname-$i.pnm
echo "Finish merging whole picture... "
pnmcat -leftright $savedname-$(($i-1))-merged.pnm $savedname-$i-cut.pnm > $savedname-$i-merged.pnm
rm $savedname-$(($i-1))-merged.pnm $savedname-$i-cut.pnm
fi
else
mv $savedname-$i.pnm $savedname-$i-merged.pnm
fi
done
mv $savedname-$(($imax-1))-merged.pnm $savedname-merged.pnm
echo Done!
#pnmtopng $savedname-merged.pnm > $savedname-merged.png
Autres cartes satellites
Pour des raisons de copyright, ce script ne contient aucune référence à un site provider de cartes satellites, il est ici à titre purement éducatif et son éventuelle modification en vue d'une utilisation illicite au regard des licenses sur les cartes est sous votre entière responsabilité.
Partisans des logiciels libres, bienvenue au XXIème siècle!
#!/bin/bash
# Version 0.12 (06/08/2005) Copyright {SHA}Pqzqg/tBLZdMRSC88PlRCZD4kLA=
#TODO
## detect non zoomable areas (if size=2085 and png instead of jpeg)
# for hybrid, this is already ok: break hybrid if not available
## Use long:lat
## check nr tiles possible+wrap
###############################################################
# This should not be changed:
# Zoom is currently between 0 (max) and 17 (whole world in one tile)
ZOOMLIMIT=17
# Scale between YYY and Xxxxxx data: (experimental)
SCALE_YYY=1.046613
###############################################################
# PARAMETERS
# PREFIX:
# Simply the prefix to append to temporary and final files
# UPPERLEFT:
# This script accepts UPPERLEFT input in qrst string format (as kh.xxxxxx.com)
# or in H:V:Z format (as mt.xxxxxx.com)
# To find qrst or H:V:Z coordinates, go to maps.xxxxxx.com, satellite view,
# navigates to the desired upper-left corner, page info->media->check images
# NR_H_TILES and NR_V_TILES:
# Indicate the nr of horizontal and vertical cells to retrieve
# PROJ=[2-3]
# Projection can be Mercator projection => PROJ=3
# cf http://en.wikipedia.org/wiki/Mercator_projection
# or Plate carree (equidistant cylindral projection) => PROJ=2
# cf http://en.wikipedia.org/wiki/Plate_carr%C3%A9e_projection
# When making hybrid maps, be sure to use PROJ=3!
# Default:
# PROJ=3
# SAT_SOURCE=[XXXXXX|YYY]:
# Select source of satellite views
# Remark: YYY satellite views are already with metadata
# and the script takes directly UPPERLEFT as YYY coordinates
# so providing SAT_SOURCE=YYY + some HYBRID_* will go wrong!
# Anyway who wants their poor outdated satellite views??
# Default:
# SAT_SOURCE=XXXXXX
# HYBRID_XXXXXX=[false|true]
# HYBRID_YYY=[false|true]
# Combines satellite view with vectorized data
# Can be data from Xxxxxx or from YYY
# YYY metadata are not transparent and not aligned with Xxxxxx data
# but who cares? the script make them transparent and aligned :-)
# This is particularly interesting for non-US locations where
# Xxxxxx metadata is very poor.
# Default:
# HYBRID_XXXXXX=false
# HYBRID_YYY=false
# HYBRID_YYY_CLEAN=0x[0-F]
# Bits indicate which elements to remove:
# 1: Remove land
# 2: Remove sea
# 3: Remove sea coast
# 4: Remove inner frontiers
# Best combinations are probably 0x3, 0x7 and 0xF
# Default:
# HYBRID_YYY_CLEAN=0xF
# HYBRID_YYY_WHEN=[BEFORE_HYBRID_XXXXXX|AFTER_HYBRID_XXXXXX]
# If both metadata, in which order to apply them?
# Default:
# HYBRID_YYY_WHEN=AFTER_HYBRID_XXXXXX
# For ease of use, the script can be called for direct conversions as such:
# gmaps <qrst string>
# gmaps <horizontal int> <vertical int> <zoom>
PREFIX="/tmp/test"
UPPERLEFT=
NR_H_TILES=
NR_V_TILES=
# In case of interruption while downloading, the script should
# recover automatically.
# Even if interrupted and restarted later, it detects what is
# already done and continues from there.
#############################################################
# Examples
# World Mercator:
#ZOOM=15
#PREFIX="/tmp/world-mercator"
#UPPERLEFT=0:0:$ZOOM
#NR_H_TILES=$((2**($ZOOMLIMIT-$ZOOM)))
#NR_V_TILES=$((2**($ZOOMLIMIT-$ZOOM)))
#PROJ=3
# World Plate:
#ZOOM=15
#PREFIX="/tmp/world-plate"
#UPPERLEFT=0:$(((2**($ZOOMLIMIT-$ZOOM))/4)):$ZOOM
#NR_H_TILES=$((2**($ZOOMLIMIT-$ZOOM)))
#NR_V_TILES=$((2**($ZOOMLIMIT-$ZOOM-1)))
#PROJ=2
# Europe hybrid:
#PREFIX="/tmp/europe"
#UPPERLEFT=15:10:12
#NR_H_TILES=4
#NR_V_TILES=3
#HYBRID_XXXXXX=true
#PREFIX="/tmp/europe"
#UPPERLEFT=15:10:12
#NR_H_TILES=4
#NR_V_TILES=3
#SAT_SOURCE=XXXXXX
#HYBRID_YYY=true
# Romania hybrid:
#PREFIX="/tmp/romania"
#UPPERLEFT=142:88:9
#NR_H_TILES=8
#NR_V_TILES=6
#HYBRID_XXXXXX=true
#HYBRID_YYY=true
# Oradea:
#PREFIX="/tmp/oradea"
#UPPERLEFT=4594:2878:4
#NR_H_TILES=3
#NR_V_TILES=3
# Belgium hybrid:
#PREFIX="/tmp/belgium"
#UPPERLEFT=259:170:8
#NR_H_TILES=7
#NR_V_TILES=5
#HYBRID_XXXXXX=true
#HYBRID_YYY=true
# Liege available in max resol: !! huge file
#PREFIX="/tmp/liege"
#UPPERLEFT=trtqtqtsrrrqrtstrr
#UPPERLEFT=67499:44060:0
#NR_H_TILES=89
#NR_V_TILES=93
# Manhattan hybrid: this was only to tune the SCALE_YYY parameter
#PREFIX="/tmp/manhattan"
#UPPERLEFT=tqstqrqrrqrsqtstqq
#NR_H_TILES=2
#NR_V_TILES=2
#HYBRID_XXXXXX=true
#HYBRID_YYY=true
#HYBRID_YYY_WHEN=BEFORE_HYBRID_XXXXXX
# Canberra
PREFIX="/tmp/canberra"
UPPERLEFT=tsrrtsqsqttqt
NR_H_TILES=2
NR_V_TILES=2
#########################################################################
### HELPER FUNCTIONS
# takes a qrst string and returns h, v and z into variables $h, $v and $z
# h and v are between 0 and 2^(ZOOMLIMIT-z)
# Example: qrst2hvz tqstqrrqstsrtsqsqr;echo $h $v $z
function qrst2hvz {
qrst=$1
h=0
v=0
z=$ZOOMLIMIT
# First must be always a "t"
if [ "${qrst:0:1}" != "t" ];
then
echo "Error! QRST string does not start by \"t\": \"$1\""
exit
fi
qrst=${qrst:1}
if [ ${#qrst} -gt $ZOOMLIMIT ];
then
echo "Error! QRST string too long: \"$1\" (max is $(($ZOOMLIMIT+1)))"
exit
fi
while [ ${#qrst} -ne 0 ];
do
h=$((h*2))
v=$((v*2))
z=$((z-1))
case ${qrst:0:1} in
q)
;;
r)
h=$((h+1))
;;
s)
h=$((h+1))
v=$((v+1))
;;
t)
v=$((v+1))
;;
*)
echo "Error! QRST string malformed: \"$1\" contains \"${qrst:0:1}\""
exit
;;
esac
qrst=${qrst:1}
done
}
# takes a triplet of h v z values and returns a qrst string into variable $qrst
# Example: hvz2qrst 39637 50100 0;echo $qrst
function hvz2qrst {
h=$1
v=$2
z=$3
HVLIMIT=$((2**($ZOOMLIMIT-$z)))
qrst=""
if [ $h -gt $HVLIMIT ];
then
echo "Error! horizontal int wrong: $h (max is $HVLIMIT)"
exit
fi
if [ $v -gt $HVLIMIT ];
then
echo "Error! vertical int wrong: $v (max is $HVLIMIT)"
exit
fi
if [ $z -gt $ZOOMLIMIT ];
then
echo "Error! zoom level wrong: $z (max is $ZOOMLIMIT)"
exit
fi
local i
for ((i=0;i<$ZOOMLIMIT-$z;i++));
do
case $((h%2)):$((v%2)) in
0:0)
qrst="q$qrst"
;;
1:0)
qrst="r$qrst"
;;
1:1)
qrst="s$qrst"
;;
0:1)
qrst="t$qrst"
;;
esac
h=$((h/2))
v=$((v/2))
done
qrst="t$qrst"
}
# takes a r0123 YYY string and returns h, v and z into variables $h, $v and $z
# TODO actually no idea of the zoom limits for YYY...
# h and v are between 0 and 2^(ZOOMLIMIT-z)
# Example: yyy2hvz r01230123;echo $h $v $z
function yyy2hvz {
yyy=$1
h=0
v=0
z=$ZOOMLIMIT
# First must be always a "r" or a "h"
if [ "${yyy:0:1}" != "r" ] && [ "${yyy:0:1}" != "h" ];
then
echo "Error! YYY string does not start by \"r\" neither \"h\": \"$1\""
exit
fi
yyy=${yyy:1}
if [ ${#yyy} -gt $ZOOMLIMIT ];
then
echo "Error! YYY string too long: \"$1\" (max is $(($ZOOMLIMIT+1)))"
exit
fi
while [ ${#yyy} -ne 0 ];
do
h=$((h*2))
v=$((v*2))
z=$((z-1))
case ${yyy:0:1} in
0)
;;
1)
h=$((h+1))
;;
2)
v=$((v+1))
;;
3)
h=$((h+1))
v=$((v+1))
;;
*)
echo "Error! YYY string malformed: \"$1\" contains \"${yyy:0:1}\""
exit
;;
esac
yyy=${yyy:1}
done
}
# takes a triplet of h v z values and returns a YYY string into variable $yyy
# Example: hvz2yyy 39637 50100 0;echo $yyy
function hvz2yyy {
h=$1
v=$2
z=$3
HVLIMIT=$((2**($ZOOMLIMIT-$z)))
yyy=""
if [ $h -gt $HVLIMIT ];
then
echo "Error! horizontal int wrong: $h (max is $HVLIMIT)"
exit
fi
if [ $v -gt $HVLIMIT ];
then
echo "Error! vertical int wrong: $v (max is $HVLIMIT)"
exit
fi
if [ $z -gt $ZOOMLIMIT ];
then
echo "Error! zoom level wrong: $z (max is $ZOOMLIMIT)"
exit
fi
local i
for ((i=0;i<$ZOOMLIMIT-$z;i++));
do
yyy="$(((h%2)+((v%2)*2)))$yyy"
h=$((h/2))
v=$((v/2))
done
yyy="r$yyy"
}
#######################################################
### Some shortcuts to use the script to make conversions:
case ${#*} in
1)
string=$1
case ${string:0:1} in
t)
echo "QRST to H:V:Z conversion:"
qrst2hvz $1
echo "$1=$h:$v:$z"
;;
[rh])
echo "YYY to H:V:Z conversion:"
yyy2hvz $1
echo "$1=$h:$v:$z"
;;
*)
echo "Error! string not understood! $1"
;;
esac
exit
;;
3)
echo "H:V:Z to QRST and YYY conversion:"
hvz2qrst $1 $2 $3
hvz2yyy $1 $2 $3
echo "$1:$2:$3=$qrst=$yyy"
exit
;;
esac
#######################################################
function yyy_merge {
# Offsets between YYY and Xxxxxx data
# YYY is smaller so we have to scale it a bit
# but that means the YYY tiles become bigger than the 256px of the Xxxxxx tiles
# Both are centered in the middle
# So origin is at (2^(17-$my_z))/2
# and uppercorner tile in origin axis becomes $my_hv-(2^(17-$my_z))/2
# ...times 256 to get the uppercorner in pixels
# Offset scaler is $SCALE_YYY-1
# the last (...+0.5)/1 is just a trick to round the result
echo "Getting YYY metadata... "
xoff=$(echo "((($my_h-(2^($ZOOMLIMIT-$my_z-1)))*256*($SCALE_YYY-1))+0.5)/1"|bc)
yoff=$(echo "((($my_v-(2^($ZOOMLIMIT-$my_z-1)))*256*($SCALE_YYY-1))+0.5)/1"|bc)
my_x=$(echo "$my_h-($xoff/(256*$SCALE_YYY))"|bc)
my_y=$(echo "$my_v-($yoff/(256*$SCALE_YYY))"|bc)
if [ $xoff -gt 0 ];
then
my_x=$(($my_x-1))
fi
if [ $yoff -gt 0 ];
then
my_y=$(($my_y-1))
fi
NR_H_TILES=$(($NR_H_TILES+1))
NR_V_TILES=$(($NR_V_TILES+1))
NR_TILES=$(($NR_H_TILES*$NR_V_TILES))
for ((j=0;j<$NR_V_TILES;j++));
do
jj=$(($my_y+$j))
for ((i=0;i<$NR_H_TILES;i++));
do
ii=$(($my_x+$i))
tile=$PREFIX-$ii-$jj-$my_z
hvz2yyy $ii $jj $my_z
request="http://tiles${yyy:${#yyy}-1}.virtualearth.yyy.com/tiles/r${yyy:1}.png?g=1"
get_hybrid $tile-yyy.png YYY
if [ "$HYBRID_YYY" != "true" ]; # shortcut to break if error 404
then
break
fi
pngtopnm $tile-yyy.png > $tile-yyy-orig.pnm
#rm $tile-yyy.png
done
if [ "$HYBRID_YYY" != "true" ]; # shortcut to break if error 404
then
break
fi
echo "Merge images in row..."
pnmcat -leftright $(for ((ii=$my_x;ii<$my_x+$NR_H_TILES;ii++)); do echo -n " $PREFIX-$ii-$jj-$my_z-yyy-orig.pnm";done) > $PREFIX-$j-up_merge-yyy-orig.pnm
rm -f $PREFIX-*-$jj-$my_z-yyy-orig.pnm
done
if [ "$HYBRID_YYY" != "true" ]; # shortcut to break if error 404
then
return
fi
echo "Merge rows... "
pnmcat -topbottom $(for ((j=0;j<$NR_V_TILES;j++)); do echo -n " $PREFIX-$j-up_merge-yyy-orig.pnm";done) > $PREFIX-yyy-orig.pnm
rm -f $PREFIX-*-up_merge-yyy-orig.pnm
echo "Merging with YYY metadata... "
# Colors to remove from the YYY map:
HYBRID_YYY_CLEAN=${HYBRID_YYY_CLEAN:-0xF}
clean_map=""
# Land:
if [ $(($HYBRID_YYY_CLEAN & 0x1)) -ne 0 ];
then
clean_map=$clean_map"F6/F1/E6 "
fi
# Sea:
if [ $(($HYBRID_YYY_CLEAN & 0x2)) -ne 0 ];
then
clean_map=$clean_map"B3/C6/D4 "
fi
# Littoral:
if [ $(($HYBRID_YYY_CLEAN & 0x4)) -ne 0 ];
then
clean_map=$clean_map"C4/D4/DC B4/CC/D4 E3/E4/DD B3/C6/D5 DA/DC/DA F4/EC/E4 BC/CC/D3 EC/EC/E4 CC/DC/DC CC/D4/DC BC/CC/DC "
fi
# Frontiers:
if [ $(($HYBRID_YYY_CLEAN & 0x8)) -ne 0 ];
then
clean_map=$clean_map"FC/F3/9B F7/DD/A1 F4/EC/D4 F4/EC/DC F4/EC/CC F4/EC/C0 FA/E9/7E EC/D1/71 E0/B4/66 D6/A3/5D 8D/6E/4D 96/8C/5F 57/50/42 EF/DB/92 FC/EC/CC F2/EC/DF 51/42/36 F9/D0/89 "
fi
# Construct call to ppmchange:
local tmp_ppmchange
tmp_ppmchange=""
local k
for k in $clean_map;do
tmp_ppmchange=$tmp_ppmchange"rgb:$k black "
done
ppmchange -remainder white $tmp_ppmchange $PREFIX-yyy-orig.pnm |\
ppmtopgm|pgmtopbm|pbmclean|pbmtopgm 5 5 > $PREFIX-yyy-mask.pgm
#without smooth: ppmtopgm > $PREFIX-yyy-mask.pgm
pnmscale $SCALE_YYY $PREFIX-yyy-orig.pnm > $PREFIX-yyy-orig-scaled.pnm
pnmscale $SCALE_YYY $PREFIX-yyy-mask.pgm > $PREFIX-yyy-mask-scaled.pgm
yoff=$(echo "(($yoff-(256*$SCALE_YYY*($my_v-$my_y)))+0.5)/1"|bc)
xoff=$(echo "(($xoff-(256*$SCALE_YYY*($my_h-$my_x)))+0.5)/1"|bc)
pnmcomp -xoff=$xoff -yoff=$yoff -alpha=$PREFIX-yyy-mask-scaled.pgm $PREFIX-yyy-orig-scaled.pnm $PREFIX.pnm > $PREFIX-tmp.pnm
mv $PREFIX-tmp.pnm $PREFIX.pnm
rm $PREFIX-yyy-orig.pnm $PREFIX-yyy-mask.pgm $PREFIX-yyy-orig-scaled.pnm $PREFIX-yyy-mask-scaled.pgm
}
function get_hybrid {
file=$1
if [ ! -s $file ];
then
echo "Getting mask $(($i+$j*$NR_H_TILES+1))/$NR_TILES @ x=$ii y=$jj"
until result=$(wget --tries=inf --no-verbose "$request" --output-document=$file 2>&1)
do
if echo $result|grep "403";then
echo "!!!INTERRUPT: We apparently got ourselves blacklisted!!!"
exit
fi
if echo $result|grep "404";then
eval "HYBRID_$2=false"
rm $file
break
fi
sleep 1;
echo "Retry...";
done
fi
}
case $UPPERLEFT in
# This is not an exhaustive check on the UPPERLEFT, just first char.
[rh]*)
yyy2hvz $UPPERLEFT
hvz2qrst $h $v $z
QRST_UPPERLEFT=$qrst
;;
t*)
QRST_UPPERLEFT=$UPPERLEFT
;;
[0-9]*:[0-9]*:[0-9]*)
hv=${UPPERLEFT%:*}
hvz2qrst ${hv%:*} ${hv#*:} ${UPPERLEFT##*:}
QRST_UPPERLEFT=$qrst
;;
*)
echo "Error! UPPERLEFT malformed: $UPPERLEFT"
exit
;;
esac
qrst2hvz $QRST_UPPERLEFT
my_h=$h
my_v=$v
my_z=$z
NR_TILES=$(($NR_H_TILES*$NR_V_TILES))
# By default, no metadata
HYBRID_XXXXXX=${HYBRID_XXXXXX:-false}
HYBRID_YYY=${HYBRID_YYY:-false}
# Print some stats
echo "Get map h=$my_h-$(($my_h+$NR_H_TILES-1)) v=$my_v-$(($my_v+$NR_V_TILES-1)) z=$my_z"
echo "This will produce a $((256*$NR_H_TILES))x$((256*$NR_V_TILES)) pixels map"
if [ "$HYBRID_XXXXXX" != "false" ];then
echo "Map will be hybrid i.e. merging of satellite and vector data from Xxxxxx"
fi
if [ "$HYBRID_YYY" != "false" ];then
echo "Map will be hybrid i.e. merging of satellite and vector data from YYY"
fi
JPGSIZE=$((12500*$NR_TILES/1024))
if [ $JPGSIZE -gt 1024 ];then
JPGSIZE="$(($JPGSIZE/1024)) Mb"
else
JPGSIZE="$JPGSIZE kb"
fi
echo "Size of jpg file (~= size of the $NR_TILES tiles to download): around $JPGSIZE"
# Required diskspace: to stock twice the pnm version of the big map (when last merge occurs)
# + the final jpeg + the original jpeg bits if not deleted at each round
# TODO: this does not count yet space required for metadata HYBRID_*
echo "Required temporary diskspace: around $(((2*(196623+12500)*$NR_TILES/1024/1024)+1)) Mb"
if [ $NR_TILES -gt 500 ];
then
echo "WARNING! Xxxxxx will probably blacklist you for a while"
echo "because of the number of downloads ($NR_TILES)"
fi
echo "Are you sure to proceed? (y/n)"
read
[ "$REPLY" != "y" ] && exit
### Recovery:
# If old half-merged files are found, try to use them
MERGE=0
if [ -s $PREFIX-*-up_merge.pnm ];
then
MERGE=$(ls -1 $PREFIX-*-up_merge.pnm|tail -1)
echo "Found $MERGE, try to recover from there..."
MERGE=${MERGE##$PREFIX-}
MERGE=${MERGE%%-up_merge.pnm}
MERGE=$(($MERGE+1))
if [ $MERGE -gt $(($NR_H_TILES)) ];
then
echo "$MERGE not in range of 1-$(($NR_H_TILES))! Discard..."
MERGE=0
fi
fi
###############################################################
# By default, satellite views from Xxxxxx and Mercator projection
SAT_SOURCE=${SAT_SOURCE:-XXXXXX}
PROJ=${PROJ:-3}
rm -f $PREFIX-*-*-$my_z.pnm
for ((j=0;j<$NR_V_TILES;j++));
do
# Recovering?
if [ $MERGE -ne 0 ];
then
j=$MERGE
MERGE=0
fi
jj=$(($my_v+$j))
for ((i=0;i<$NR_H_TILES;i++));
do
ii=$(($my_h+$i))
tile=$PREFIX-$ii-$jj-$my_z
# Build request depending on the source, Xxxxxx by default
case "$SAT_SOURCE" in
XXXXXX)
hvz2qrst $ii $jj $my_z
request="http://kh.xxxxxx.com/kh?v=$PROJ&t=$qrst"
;;
YYY)
hvz2yyy $ii $jj $my_z
request="http://tiles${yyy:${#yyy}-1}.virtualearth.yyy.com/tiles/h${yyy:1}.jpeg?g=1"
;;
esac
if [ ! -s $tile.jpg ];
then
echo "Getting bitmap $(($i+$j*$NR_H_TILES+1))/$NR_TILES @ x=$ii y=$jj"
until result=$(wget --tries=inf --no-verbose "$request" --output-document=$tile.jpg 2>&1)
do
if echo $result|grep "403";then
echo "!!!INTERRUPT: We apparently got ourselves blacklisted!!!"
exit
fi
sleep 1;
echo "Retry...";
done
fi
echo -n "Converting to PNM... "
jpegtopnm $tile.jpg > $tile.pnm
#rm $tile.jpg
# Build request depending on the source of meta, none by default
if [ "$HYBRID_XXXXXX" == "true" ];
then
hvz2qrst $ii $jj $my_z
# for info, transparent is with v=w2t.1 and plain is with v=w2.5
request="http://mt.xxxxxx.com/mt?v=w2t.1&x=$ii&y=$jj&zoom=$my_z"
get_hybrid $tile-xxxxxx.png XXXXXX
if [ "$HYBRID_XXXXXX" == "true" ];
then
pngtopnm $tile-xxxxxx.png > $tile-xxxxxx-orig.pnm
pngtopnm -alpha $tile-xxxxxx.png > $tile-xxxxxx-mask.pgm
#rm $tile-xxxxxx.png
fi
fi
done
echo "Merge images in row..."
pnmcat -leftright $(for ((ii=$my_h;ii<$my_h+$NR_H_TILES;ii++)); do echo -n " $PREFIX-$ii-$jj-$my_z.pnm";done) > $PREFIX-$j-up_merge.pnm
rm -f $PREFIX-*-$jj-$my_z.pnm
if [ "$HYBRID_XXXXXX" == "true" ];
then
pnmcat -leftright $(for ((ii=$my_h;ii<$my_h+$NR_H_TILES;ii++)); do echo -n " $PREFIX-$ii-$jj-$my_z-xxxxxx-orig.pnm";done) > $PREFIX-$j-up_merge-xxxxxx-orig.pnm
pnmcat -leftright $(for ((ii=$my_h;ii<$my_h+$NR_H_TILES;ii++)); do echo -n " $PREFIX-$ii-$jj-$my_z-xxxxxx-mask.pgm";done) > $PREFIX-$j-up_merge-xxxxxx-mask.pgm
rm -f $PREFIX-*-$jj-$my_z-xxxxxx-orig.pnm $PREFIX-*-$jj-$my_z-xxxxxx-mask.pgm
fi
done
echo "Merge rows... "
pnmcat -topbottom $(for ((j=0;j<$NR_V_TILES;j++)); do echo -n " $PREFIX-$j-up_merge.pnm";done) > $PREFIX.pnm
rm $PREFIX-*-up_merge.pnm
if [ "$HYBRID_XXXXXX" == "true" ];
then
pnmcat -topbottom $(for ((j=0;j<$NR_V_TILES;j++)); do echo -n " $PREFIX-$j-up_merge-xxxxxx-orig.pnm";done) > $PREFIX-xxxxxx-orig.pnm
pnmcat -topbottom $(for ((j=0;j<$NR_V_TILES;j++)); do echo -n " $PREFIX-$j-up_merge-xxxxxx-mask.pgm";done) > $PREFIX-xxxxxx-mask.pgm
rm -f $PREFIX-*-up_merge-xxxxxx-orig.pnm $PREFIX-*-up_merge-xxxxxx-mask.pgm
fi
# Merging with metadata, by default if both Xxxxxx and YYY, Xxxxxx first:
HYBRID_YYY_WHEN=${HYBRID_YYY_WHEN:-AFTER_HYBRID_XXXXXX}
if [ "$HYBRID_YYY" == "true" ] && [ "$HYBRID_YYY_WHEN" == "BEFORE_HYBRID_XXXXXX" ];
then
yyy_merge
fi
if [ "$HYBRID_XXXXXX" == "true" ];
then
echo "Merging with Xxxxxx metadata... "
pnmcomp -alpha=$PREFIX-xxxxxx-mask.pgm $PREFIX-xxxxxx-orig.pnm $PREFIX.pnm > $PREFIX-tmp.pnm
mv $PREFIX-tmp.pnm $PREFIX.pnm
rm $PREFIX-xxxxxx-orig.pnm $PREFIX-xxxxxx-mask.pgm
fi
if [ "$HYBRID_YYY" == "true" ] && [ "$HYBRID_YYY_WHEN" == "AFTER_HYBRID_XXXXXX" ];
then
yyy_merge
fi
#echo Converting to png format...
#pnmtopng $PREFIX.pnm > $PREFIX.png
echo Converting to jpeg format...
pnmtojpeg $PREFIX.pnm > $PREFIX.jpg
rm $PREFIX.pnm
echo Done! File is $PREFIX.jpg
Pages blanches
Annuaire inversé en ligne de commande:
#!/bin/bash
#Usage:
#>pages 02 1234567
### Ancien site, hors service
#lynx -dump "http://www.belgacom.be/cgi-bin/whitePages?L=F&TPL=IP2&BYNUMBER=YES&P1=$1&P2=$2"|gawk '
#/Votre recherche/{exit}
#body==1
#/Résultats/{body=1}
#'
wget -O - -q --header="Cookie: language_cookie=fr;" \
"http://www.skynet.be/index.html?l1=search&l2=whitepages&l3=reverse\
&prefixe=$1&ntel=$2&search=Rechercher"\
|gawk '\
/Résultats de la recherche/ {\
while ($0 !~ /<td class=.content_text/) getline; \
getline;\
while ($0 !~ /<!--/){\
gsub(/^[[:space:]]*|<\/?[a-z]*>/,"");\
print;\
getline\
}\
}\
'
Volume mensuel consommé chez Skynet ADSL
#!/bin/bash LOGIN=<YOURLOGIN> PASSW=<YOURPASSW> #Change CFID & CFTOKEN to have a unique one CFID=1234567 CFTOKEN=12345678 echo -n "fuseaction=CheckLoginConnection&form_login=$LOGIN&form_password=$PASSW&Langue_Id=1&Submit=Inloggen"\ |POST -t 2 -H "Cookie: CFID=$CFID; CFTOKEN=$CFTOKEN" -d\ https://e-care.skynet.be/Login/index.cfm wget -O /dev/null --quiet\ --header="Cookie: CFID=$CFID; CFTOKEN=$CFTOKEN"\ 'https://e-care.skynet.be/Login/index.cfm?fuseaction=LoginConnection&NextPage=../App/ConTime/index.cfm?fuseaction=VolumeMeter' wget -O - --quiet\ --header="Cookie: CFID=$CFID; CFTOKEN=$CFTOKEN"\ 'https://e-care.skynet.be/App/ConTime/index.cfm?fuseaction=VolumeMeter'\ |grep disponibles |sed 's/<[^<]*>//g;s/^\t*//'
La sortie ressemblera à ceci:
- Volume mensuel utilisé 3.75 Gb sur 10 Gb disponibles.
Dépendances:
- libwww-perl
- libcrypt-ssleay-perl
- wget-ssl
pour debugger : livehttpheaders de firefox ;-)
Vérification automatique des Free Skype Days
Le but est de se logguer sur son compte Skypeout et de vérifier si ce jour est un Free Skype Day.
Ce script permet de se faire la main avec les lwp-request (GET, POST,...) disponibles dans le paquet libwww-perl.
#!/bin/bash
USER="<my_user>"
PASS="<my_pass>"
LOG="/var/log/skypeout.txt"
# Be sure to have write access to the log file:
touch $LOG||exit 1
# Login
# First: get a session_id
session_id=$(\
echo -n "username=$USER&password=$PASS&login=Sign+me+in"|\
POST -e -H 'Host: secure.skype.com' \
'https://secure.skype.com/store/member/dologin.html'|\
grep Set-Cookie|sed 's/.*\([a-f0-9]\{32\}\).*/\1/'\
)
# Then try again with this session_id
echo -n "username=$USER&password=$PASS&login=Sign+me+in"|\
POST -H 'Host: secure.skype.com' \
-H "Cookie: skype_store2=$session_id; username=$USER" \
'https://secure.skype.com/store/member/dologin.html'
# Get overview
results=$(\
GET -H 'Host: secure.skype.com' \
-H "Cookie: skype_store2=$session_id; username=$USER; loggedin=1" \
'https://secure.skype.com/store/myaccount/overview.html'|grep "Free Skype Day" \
)
# Noe that we've collected the data, this part is up to you, depends what you want to do
date >> $LOG
echo $results >> $LOG
echo "--------------------------------" >>$LOG
subject=$(echo $results |gawk '{gsub(/<[^>]*>/,"");gsub(/&#[0-9]+;/,"");print;exit}')
echo $subject|grep -q "Today is not Free Skype Day"||echo $results|mail -s "$subject" <myself>
# Logout
GET -H 'Host: secure.skype.com' \
-H "Cookie: skype_store2=$session_id; username=$USER; loggedin=1" \
'https://secure.skype.com/store/member/logout.html' >/dev/null 2>&1