Einfache Sitemaps erstellen

Viele Sitemapgeneratoren arbeiten nur direkt von der Shell des Webservers. Hier ein Script, welches auch von extern per wget arbeitet und damit eine sitemap.xml erstellt.

Das Bash-Script

mksitemap.sh
(PDF-Datei mit Quelltext)

Das Script kopiert man z.B. nach

und macht es mit

ausführbar. Danach kann man es mit

aufrufen.
Eine Finetuning-Funktion ist eingebaut: man kann bei Schlüsselwörtern in der URL ohne Probleme eigene Prioritäten vergeben, die die Suchmaschine vielleicht teil-mitbeachtet.

Google und vor allem Yahoo! bieten sehr viele interessante Services für Webmaster, die man sich mal anschauen sollte. Die Links folgen im nächsten Abschitt.

Quellcode der PDF-Datei

#/bin/bash

#

# mksitemap.sh - create sitemaps by crawling a site using wget

# Copyright (C) 2010 Benjamin M.

#

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program.  If not, see <http://www.gnu.org/licenses/>.



############################

##   F U N C T I O N S    ##

############################



# downloads most important files

# doesn't work with no filenames

function geturl 

{



        wget --spider -m -nv -A "*.html,*.htm,*.php,*.txt" -o wget.log "$1" # -l 1



        if [ 0 -ne $? ]; then

                echo "Konnte Adresse $1 nicht auflösen." 2>/dev/stderr

                exit 2

        fi

}



# grep and parse URLs, then put them into an array

function makelist

{

        local  -a urllist



        if [ -z "$1" ]; then

                echo "Leere URL-Liste. Beende." 2>/dev/stderr 

                exit 4

        fi



        # Aufbau: 

        # 

        urllist=(`      grep "http" wget.log  | \

                                sed 's/^.*URL:\(http:.*\)\( \[.*\[.*$\)/\1/g' | \

                                uniq | \

                                sort | \

                                xargs -0 `)

        echo ${urllist[*]}

}



# XML-File-Definition, bla

write_header() 

{

        (

        cat <<AOF

<?xml version='1.0' encoding='UTF-8'?>

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9

        http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">

AOF

        ) >> "$1"

}



# End of XML file

write_footer()

{

        (

        cat <<EOF

</urlset> 

EOF

        ) >> "$1"

}



# Make some important sites more important.

# Should be changed to fit your page's URLs.

rate_url()

{

        if [[ "$1" =~ .*News*0\.html ]] ||

                [[ "$1" =~ .*News\.html ]]; then

                echo "0.9"

        elif [[ "$1" =~ .*News.* ]]; then

                echo "0.5"

        elif [[ "$1" =~ .*Termine.* ]]; then

                echo "0.8"

        elif [[ "$1" =~ .*Kontakt.* ]]; then

                echo "0.2"

        elif [[ "$1" =~ .*Fotoalbum.* ]]; then

                echo "0.6"

        else

                echo "0.4"

        fi

}



show_version()

{

(

        cat <<-EOV

        mksitemaps Copyright (C) 2010 Benjamin M.

        This program comes with ABSOLUTELY NO WARRANTY; 

        This is free software, and you are welcome to redistribute it

        under certain conditions;



EOV

) > /dev/stderr





}





################

#  M  A  I  N  #

################



# Variablen

declare -a urllist

priority=0

wgetlogfile=wget.log

sitemapfile=sitemap.xml



show_version



# Parameter

if [ -z $1 ]; then

        echo "Keine URL angegeben - beende." 2>/dev/stderr

        exit 1

fi



if [ -f "$wgetlogfile" ]; then

        rm -f "$wgetlogfile"

fi

if [ -f "$sitemapfile" ]; then

        rm -f "$sitemapfile"

fi



geturl "$1" 

urllist=(`makelist "$wgetlogfile"`)



write_header "$sitemapfile"



i=0

while [ $i -lt ${#urllist[@]} ]; do

        priority=`rate_url "${urllist[$i]}"`

        echo -ne "Parsing link #"`echo -n $((i+1))`" von ${#urllist[@]}: [$priority] ${urllist[$i]}\n" > /dev/stderr

        echo -ne "\t<url>\n" >> "$sitemapfile"

        echo -ne "\t\t<loc>${urllist[$i]}</loc>\n" >> "$sitemapfile"

        echo -ne "\t\t<priority>$priority</priority>\n" >> "$sitemapfile"

        echo -ne "\t</url>\n" >> "$sitemapfile"

        let i++

done



write_footer "$sitemapfile"



rm -f wget.log



exit 0

Weiterführende Links

• Wikipediaeintrag über Sitemaps
  http://de.wikipedia.org/wiki/Sitemaps
• Googles Sitemap-Generator (braucht Shell-Zugriff)
  http://code.google.com/p/sitemap-generators/
• einfacher Online Sitemap-Generator
  http://www.xml-sitemaps.com/
• Sitemap bei Google Webmaster Tools einreichen inkl. Check
  http://www.google.com/webmasters/sitemaps/?hl=de
• Sitemap bei Yahoo! einreichen
  https://siteexplorer.search.yahoo.com/submit

Ähnliche Beiträge:

Videos für das Handy umwandeln Minimales Pydev: Schnell und schlank Dropbox vs. myDrive