Mediawiki RawFile

Introduction

Originally the idea was to be able to download directly a portion of code as a file.
I've numerous code examples in my wiki and I wanted an easy way to download them, easier than a copy/paste!
But from there it was rather easy to get something very close to literate programming just by allowing multiple blocks referring to the same file, which will be concatenated together at download time.

It must work with pre, nowiki, js, css, code, source, so let's make it general: take the tag that comes after the parser function we'll create and select data up to the closing tag.
There are two distinct functionalities provided by the extension:
- the parser that will convert a magic word into a link to the download URL
- an extended ?action=raw that will strip the raw output to keep the desired code

Documentation

Syntax

There are 2 kinds of elements to add to the wiki language:

anchors that will flag which code blocks belong to a specific file
- {{#rawsnippetAnchor: myscript.sh}}
- Not visible in the regular wiki display
links that will allow to download the file
- {{#rawsnippetLink: myscript.sh}}
- Transformed into real URLs:
  {{fullurl:{{PAGENAME}}|action=raw&rawsnippet=myscript.sh}}

For regular use, when a single code block is used and when the download link can be at the same position as the anchor, there is a shortcut notation mixing both anchor & link properties: {{#rawsnippet: myscript.sh}}

Short example

Let's save the following code [{{#rawsnippet: myscript.sh}} as myscript.sh]
<source lang=bash>
#!/bin/bash

echo 'Hello world!'
exit 0
</source>

will give:

Let's save the following code [{{#rawsnippet: myscript.sh}} as myscript.sh]

#!/bin/bash

echo 'Hello world!'
exit 0

Complete example

And a full example with anchors & link:

Let's start with the Bash usual header:
{{#rawsnippetanchor: myotherscript.sh}}
<source lang=bash>
#!/bin/bash
</source>
Then we'll display a welcome message:
{{#rawsnippetanchor: myotherscript.sh}}
<source lang=bash>
echo 'Welcome on earth!'
exit 0
</source>
[{{#rawsnippetlink: myotherscript.sh}} myotherscript.sh is now available for download below the code]

will give:

Let's start with the Bash usual header: {{#rawsnippetanchor: myotherscript.sh}}

#!/bin/bash

Then we'll display a welcome message: {{#rawsnippetanchor: myotherscript.sh}}

echo 'Welcome on earth!'
exit 0

[{{#rawsnippetlink: myotherscript.sh}} myotherscript.sh is now available for download below the code]

The code

Which you can of course download just by following [{{#rawsnippetlink: RawSnippet.php}} this link :-)]

So let's explain a bit the code in a Literate Programming way...

Hooks

First some hooks for our functions... {{#rawsnippetanchor: RawSnippet.php}}

<?php

if (defined('MEDIAWIKI')) {

# Define a setup function
$wgExtensionFunctions[] = 'efRawSnippet_Setup';
# Add a hook to initialise the magic words
$wgHooks['LanguageGetMagic'][]       = 'efRawSnippet_Magic';
# Add a hook to intercept the raw output
$wgHooks['RawPageViewBeforeOutput'][] = 'fnRawSnippet_Strip';

Setup function

function efRawSnippet_Setup() {
    global $wgParser;
    # Set a function hook associating the "rawsnippet" magic word with our function
    $wgParser->setFunctionHook( 'rawsnippet', 'efRawSnippet_Render' );
    $wgParser->setFunctionHook( 'rawsnippetlink', 'efRawSnippet_Render' );
    $wgParser->setFunctionHook( 'rawsnippetanchor', 'efRawSnippet_Empty' );
}

Hook to initialize the magic words

function efRawSnippet_Magic( &$magicWords, $langCode ) {
    # Add the magic word
    # The first array element is case sensitive, in this case it is not case sensitive
    # All remaining elements are synonyms for our parser function
    $magicWords['rawsnippet'] = array( 0, 'rawsnippet', 'downloadAs' );
    $magicWords['rawsnippetlink'] = array( 0, 'rawsnippetlink', 'downloadLink' );
    $magicWords['rawsnippetanchor'] = array( 0, 'rawsnippetanchor', 'downloadAnchor' );
    # unless we return true, other parser functions extensions will not get loaded.
    return true;
}

Parser functions of the magic words

The transformation rule to replace link shortcuts to actual links for download {{#rawsnippetanchor: RawSnippet.php}}

function efRawSnippet_Render( &$parser, $filename = '') {
    # The parser function itself
    # The input parameters are wikitext with templates expanded
    # The output should be wikitext too
    return '{{fullurl:{{PAGENAME}}|action=raw&rawsnippet='.$filename.'}}';
//TODO+support for other pages
}

And the other one, just removing the anchors from the rendered wiki page {{#rawsnippetanchor: RawSnippet.php}}

function efRawSnippet_Empty( &$parser, $filename = '') {
    return '';
}

Hook to intercept the raw output

Must extract the right paragraph
- Strip all up to the right rawsnippet: filename tag
- Find the next tag
- Select up to the closure tag
Must provide the filename to the browser
Tells the browser NOT to cache the raw

Then the function to strip the code out of the raw wiki page, quite heavy as we've to parse the wiki page ourselves... {{#rawsnippetanchor: RawSnippet.php}}

function fnRawSnippet_Strip(&$rawPage, &$text) {
    // if our ext wasn't used, just exit
    if (!isset($_GET['rawsnippet']))
        return true;
    $filename=$_GET['rawsnippet'];
    header("Content-disposition:filename=".$filename);
    header("Content-type:application/octetstream"); 
    header("Content-Transfer-Encoding: binary"); 
    header("Expires: 0");
    header("Pragma: no-cache"); 
    header("Cache-Control: no-store");
    // First: find the right anchor
    // We mask nowiki sections
//TODO also mask source, js, css, pre, what else?
    //test2 will contain interpretable content, all static content is blanked out
    $maskedtext=preg_replace_callback('/<nowiki>(.*?)<\/nowiki>/', 
        create_function(
           '$matches',
           'return ereg_replace(".","X",$matches[0]);'
        ),
        $text);
    // We search rawsnippet anchor position
    if (preg_match_all('/{{#rawsnippetanchor: +'.$filename.' *}}/i', $maskedtext, $matches, PREG_OFFSET_CAPTURE))
        $offsets=$matches[0];
    else if (preg_match_all('/{{#rawsnippet: +'.$filename.' *}}/i', $maskedtext, $matches, PREG_OFFSET_CAPTURE))
        // If the shortcut "rawsnippet" is used, no nuweb, just the first hit is considered
        $offsets=array($matches[0][0]);
    else
        // We didn't find our anchor, let's output all the raw...
// TODO change headers & send error msg
        return true;
    // free some mem
    unset($maskedtext);
    $textorig=$text;
    $text='';
    foreach ($offsets as $offset) {
        // Now let's remove the text up to our anchor
        $out = substr($textorig, $offset[1]);
        // What's the type of tag do we have?
        $out = substr($out, strpos($out, '<'));
        if (!preg_match('/^<([^> ]+)/', $out, $matches))
// TODO send error, we could not find end of bloc
            return true;
        $key = $matches[1];
        // Let's extract the text up to the closing tag
        $begin = strpos($out, '>')+1;
        if (ord(substr($out,$begin,1))==10)
            $begin++;
        if (preg_match_all('/<\/'.$key.'>/', $out, $matches, PREG_OFFSET_CAPTURE))
            $text .= substr($out, $begin, $matches[0][0][1]-$begin);
        else
// TODO send error, we could not find end of bloc
            $text .= substr($out, $begin);
    }

    header("Content-Length: ".strlen($text)); 
    return true;
    //TODO: downloadAs..
}

Credits

$wgExtensionCredits['parserhook'][] = array('name' => 'RawSnippet',
                           'version' => '0.1',
                           'author' => 'Philippe Teuwen',
//                         'url' => 'http://www.mediawiki.org/wiki/Extension:LocalServer',
                           'url' => 'http://wiki.yobi.be/wiki/Mediawiki_RawSnippet',
                           'description' => 'Downloads a RAW copy of <nowiki><tag>data</tag></nowiki> in a file<br>'.
                                            'Useful e.g. to download an example code or a patch<br>'.
                                            'It also opens the path to [http://en.wikipedia.org/wiki/Literate_programming Literate Programming]');
}

?>

Installation

Download [{{#rawsnippetlink: RawSnippet.php}} RawSnippet.php] and save it under the MediaWiki directory as extensions/RawSnippet/RawSnippet.php

Add at the end of LocalSettings.php:

require_once("$IP/extensions/RawSnippet/RawSnippet.php");