I just wrote a script to easily download your FULL friendfeed stream. I mean: every post with comments, likes and even images and/or files. Enough for the good news: the bad news is that this a PHP command line script and you have to know how to run it. On a recent  OSX or Linux environment you should be ready to go. For Windows-ers it could be a little more difficult. Sorry.

Anyway, here is the features list:

• can fetch a private stream, giving your remote key
• locally downloads images and files (detects already downloaded things)
• upon subsequent runs, it tries to get only the newly added posts
• the output file is JSON
• it scales well!
• it’s free software (MIT license)

Download the scripts from Github (the link points to the project page, where you can read the README too. In this case you’re interested to the ffexp.php script).

You then may run it like:

php ffexp.php somefilename.json

As you may infer from the output file extension, the output is plain good JSON. This format is very handy for whatever thing you may need to do with the export file. But this is up to you (or your programmer friend).

As an example of JSON postprocessor I also wrote a converter that given your export file will render a fully functional HTML file, looking just like a Friendfeed page. It also automatically embeds the necessary JavaScript, CSS and even icon images. Download the ffexp2html from Github, open it, read the fine documentation and run it like:

php ffexp2html.php somefilename.json > somefilename.html

Now you can open the somefile.html file with your browser (no internet connection needed). You may find a generated output here.

Limits? Yes there are:

• No download of your DMs (don’t know exactly why, maybe a limit of the API. But I don’t care)
• You can download even entire rooms (just use the room name as the username), but it seems that there is a limit of how back you can go. Very high, but it exists.

I wrote a little script in PHP that converts your bookmarks file exported from Delicious in some other formats. The script, if you’re in the mood, is even easily extensibile if you provide it another “driver” to convert in another format (you write it and then plug it into the script. Pretty straightforward).

The script is ready to be used if you need to convert the file in CSV.

Run it like this:

php deliconv.php delicious_export.html > delicious_export.csv

Another idea on how to use the script would be to check every URL for its existance. It’s a pretty easy task if you know how to use the curl extension. I’ll leave it as an exercise for the reader :)

So, if know how to run PHP from the command line, grab the script here (remove the .txt extension).

There is also a Delicious XML exporter, out there.

Update: my friend Nicola D’Agostino has collected some other useful informations and tricks on the matter.

As you may imagine this scraping process is tedious to set up and prone to errors in the medium/long period. I have to read through the HTML of the page and try to find the patterns that must be interpreted to obtain the needed information (as for the moment, the bots rely on a regular expression match system). But what if the project owner(s) redesign the site? Yeah, the bot miserably fails and calls for help (me, as usual). For example: recently Apache added an “(httpd)” string to the release page of its httpd server, and for the 2.2 version only :) The 2.0 branch still has no “httpd” string in it…

Now I’m wondering if a system could be implemented for the project site webmaster to give a more reliable system to give project like mine a way to gather this (all in all) very little piece of information they need.

What could be the best way to describe and publish the information about VERSION_ID, RELEASE_DATE, CHANGELOG, DOWNLOAD_URI for a project? What do you suggest? This is a short list of ideas, so far:

• META element(s) on the home page HTML
• HTTP header(s) to be requested via HTTP HEAD method
• a common, stated web service via a REST URI like http://someproject.info/info
• an RSS dialect
• a “standard” already exists that I’m not aware of :)

Your comment would be very appreciated :)

Updates:

In the comments void points out the DOAP project. Really interesting.

With the recent release of the 1.10 version of the  Zend Framework, they made a subtle change on how Zend_Tool searches its providers. Before 1.10 the loader (aka the provider discover) was set to be the IncludePathLoader class; what that meens is that if you wrote a new provider, all that you needed to do to have Zend_Tool automatically find it was to edit your PHP’s include_path or add your provider’s directory to the ZEND_TOOL_INCLUDE_PATH_PREPEND environment variable.

Now that is history, because (source):

There were many issues for people when Zend_Tool used a scanning approach to finding providers. This caused many issues on all different platforms. Now we’ve opted to go the specify your providers approach.

(beware: the suggested solution in that post is wrong)

So if you ever write a new tool provider remember that there’s no more “auto discovery” by scanning the path. The loader now is the BasicLoader and you have to explicitely tell Zend_Tool where your providers are and how their classes are named. For this to happen, you can use the zf enable config.provider command or use the zf.ini file.

My solution is:

• create a zf.ini file for your project. This is slighlty different from what the documentation implies; it considers zf.ini to be an hidden file in your $HOME. But this is only a default you can change via the  ZF_CONFIG_FILE env variable
• put something like basicloader.classes.0 = “Migrations_MigrationProvider” as the first line (that example is the actual class name of my migration tool provider)
• assure that your class can be loaded, setting ZEND_TOOL_INCLUDE_PATH_PREPEND accordingly

On the same topic you can also read this issue.

As a bonus track, below is a little bash script I use to run my migration tool.

#!/bin/bash

APPPATH=$(readlink -f ..)/app

export ZF_CONFIG_FILE=${APPPATH}/../zf.ini

if [ ! -f ${ZF_CONFIG_FILE} ]; then
  echo "Non trovo zf.ini"
  echo "Forse non stai eseguendo questo programma dalla directory DB?"
  exit -1
fi

if [[ "${ZF_BIN_DIR}" == "" ]]; then
  ZF_BIN_DIR=$(readlink -f ../vendor/Zend)/../../bin
  ZF_BIN_DIR=$(readlink -f ${ZF_BIN_DIR})
fi

if [ ! -f ${ZF_BIN_DIR}/zf.sh ]; then
  echo "Non trovo zf.sh in " ${ZF_BIN_DIR}
  echo "Forse non stai eseguendo questo programma dalla directory DB?"
  exit -1
fi

MIGCLASSDIR=$(readlink -f ../vendor/Renomo/library)

if [ ! -d ${MIGCLASSDIR} ]; then
  echo "Impossibile trovare la directory della classe Migration"
  exit -1
fi

MIGDIR=$(readlink -f migrations)

if [ ! -d ${MIGDIR} ]; then
  echo "Impossibile trovare la directory delle migration"
  exit -1
fi

export ZEND_TOOL_INCLUDE_PATH_PREPEND=${MIGCLASSDIR}

${ZF_BIN_DIR}/zf.sh run migration ${1}

È appena uscito OpenOffice 3.2. Qui trovate la lista di tutte le novità.

Gli aggiornamenti che mi interessano maggiormente sono:

• migliorato il supporto al formato ODF (che incidentalmente sarà anche ben supportato da MS Office 2010, il che garantirà finalmente un elevato grado di compatibilità tra le due suite).
• migliorato il supporto ai formati proprietari MS, diminuendo così il numero di problemi di “interfaccia” con MS Office 2007
• aggiunti i commenti su Impress (in questo modo posso facilmente lasciarmi dei post it quando faccio delle slide in bozza)
• Rispetto alla 3.0 è diminuito di un buon 46% il tempo necessario a far partire Writer o Calc (e rispetto alla 3.1? Non si sa)

Detto questo, se volete provarla subito (come io ho fatto), su un sistema Ubuntu dove magari avete già installato la 3.1 (come me), dovete fare così:

• andate a prendervi i deb di OpenOffice: li trovate qui
• mentre li scaricate in una directory di comodo, disinstallate l’attuale OOo con un bel sudo apt-get remove openoffice*
• una volta scaricati i deb, scompatatteli ed entrate nella sottodirectory DEBS. Da qui eseguite sudo dpkg -i *.deb
• entrate nella sottodirectory desktop-integration e anche qui andate di sudo dpkg -i *.deb

Dovreste essere a posto (almeno per me così è stato).

Sicuramente queste istruzioni sono meglio di niente, peccato però che OpenOffice non metta a disposizione un repository per apt… in questo modo appena uscirà la 3.2.1, saremo di nuovo punto e a capo (ma a questo punto credo che aspetterò la nuova Ubuntu che forse avrà a bordo OOo 3.2).

Ti scarichi la versione che ti interessa dal repository delle versioni di Thunderbird (che non è facile da trovare, inspiegabilmente).

Te lo scompatti da qualche parte (io uso ~/opt).

Ti accerti di avere installati i seguenti pacchetti (Ubuntu)

Te lo configuri con il seguente comando:

./configure --enable-application=mail --with-system-bz2 --with-system-zlib --with-system-jpeg --enable-calendar --disable-ldap --enab
le-official-branding --disable-accessibility --disable-ogg --disable-wave --disable-inspector-apis --disable-installer --disable-upda
ter --disable-tests --disable-parental-controls  --disable-logging --disable-necko-wifi

Ovviamente dai un’occhiata ai parametri perché potrebbero non essere tutti di tuo gradimento (hint: ./configure –help | less).

Esegui make e attendi una quindicina di minuti.

Il binario di Thunderbird te lo trovi poi in ~/opt/thunderbird-3.0/comm-1.9.1/mozilla/dist/bin/thunderbird

That is all what I needed, from Facebook: I do not need any fancy “get your friends here” or “post in your wall” rubbish. Simple and plain “single sign on”, among Facebook and my application.

After a quick search, the tutorials I found here and there were very disappointing (just a point of view, no offense), with a lot of useless PHP code. So I give you just one more tutorial that focuses more on the process than in the copy & paste kind of help.

I will use PHP for some examples, and FB for “Facebook” so bear with me and give it a spin.

What you need:

• a FB account, if you don’t own one already. Unlikely; I’ll skip this step :)
• a new FB application need to be created
• the FB PHP library, to easily talk to their RESTful API
• Some code refactoring of your own

Creating a new FB application.

Two paths, here:

• the long and detailed multistep form
• the super easy wizard

Note that, whichever way you’ll choose, you will end up creating the exact same thing: an FB application (all your applications are stored here).

We’ll use the wizard:

• the website name is just the name of the application (the handle by which you’ll refer to it in the application list).
• the website url is self explanatory. Note that you could use ANY kind of URL, even something like “http://myapplication.localdomain”. In fact, FB will not need to contact the site, so you can use an development URL as well. For the test purposes I created two applications: one for the “real thing” and one for the mydomain.local devel environment.

The next step will ask you to download a file. You can put the file WHEREVER you want. The root directory of your site is the first choice, and it’s where the wizard will try to find it. There’s no need for the wizard to find the file, though: it’s only a test. You can download the file and skip the test pressing “Upload later“. Two words about that particular file: the file is part of the system that permits our application to be proxied to FB (and vice-versa). I will not enter in details here, but think about this file as your “gateway” to and from the FB requests (more info). The other thing is: there is NOTHING in that file that identifies you application. Every FB application of the world could use the same xd_receiver.htm file. The file name itself is not important… it’s just the default. More on this later.

The third step is the landing page for your newly created FB application. Just a comfy place to pick things up. You need to write down the API KEY and the SECRET strings. You’ll use them from within your application.

What you also need to take from this page is the JavaScript snippet at point 1) and the FBML markup for the “Connect with Facebook” button at point 2).

Enough for the FB setup.

The FB PHP Library and the FB API

This is a set of helper PHP classes that will give you a simple PHP API interface to the RESTful FB API. You could even NOT use the library, and talk directly to the REST endpoints with some code of your own. Your choice.
The FB API are documented here, while you can download the latest version of the PHP 5 Library via SVN

svn co http://svn.facebook.com/svnroot/platform/clients/php/trunk/ fb

(more info on FB Client libraries)

We are finally ready to get our hands dirty.

Hands on the code

We first need to focus on how things works, at an higher level. What will happen when a not logged-in user will click the “Connect with Facebook” button? Using some sort of magic Facebook will test if the user is currently logged in Facebook. If it’s not, it will spawn a popup window asking two things: the user has to agree to give your application access to her data, and after that Facebook will ask her for credential (username/password, as usual). Upon a succesful login, FB will redirect the user to your application toward an url of your choice (let’s call it the “FB Connect endpoint“).
If the user is already logged in Facebook and has already given permission to our application to read some of her data, the login process will be completely transparent. The user clicks on our FB blue-ish button, and he will be instantly logged in our application too. Oh, joy.

As for the HTML part, you will place the FBML markup for the login button you got beforehand wherever you want. The FBML have to be programmatically converted in HTML by JavaScript, and for that to work you need to include the JavaScript snippet you already have. Put it at the end of your <body>, for example, and then run the FB.init() function. This function will be passed two parameters: the API KEY and the name and location of the “gateway” file. As said before, if the second parameter is not passed, the default is “xd_receiver.htm” on the document root.

  <!-- The login button, somewhere in your page -->
  <fb:login-button onlogin="window.location='/fbconnect.php';" v="2" size="medium">Login with Facebook</fb:login-button>
  ...
  <!-- At the page end of the page, presumibly -->
  <script src="http://static.ak.connect.facebook.com/js/api_lib/v0.4/FeatureLoader.js.php/en_US"></script>
  <script>
    FB.init("b4bbr43b4bb3b3b3");
  </script>
</body>
</html>

The interesting part, here, is in the highlighted row. The onlogin attribute contains what we pretentiously called the “FB Connect endpoint”. The onlogin attribute value is a JavaScript snippet that will be automatically run after a succesfull FB login. What we say here is: after a login, please redirect the user to our “fbconnect.php” page.
The fbconnect.php page is where OUR magic will happen. There, we will use the FB Client API to ask FB some questions: “Is this guy a Facebook, logged in user?”, “Can you please give me the picture of him?”… and so on.
The code, here, is given only as an example; think about it more as a pseudocode, really. The actual code I use is somewhat more complex, and I needed to dumb it down a little for clarity.

require("fb/facebook.php");
$fb = new Facebook("b4bbr43b4bb3b3b3", "supersecret1234");
/* If it's present, ask FB for the ID of the current user */
$fbId = $fb->get_loggedin_user();
if (empty($fbId)) {
  /* No FB user, redirect to home page */
  header("Location: /");
}
/* Now we know the FB user id.
   It's our responsability to ask our Database if we already know him */

You need to have a way to find one user of yours given its FB user id. For instance I used a “fb_id” column in my Users table.

What happens now is up to you. There will presumably be two major cases:

1. the user already exists: I programmatically login him into my application, and redirect to the home page as a logged in user.
2. the user does not exist: I give her the choice to merge her old account or to create a new one

For the merge part, I need him to login with his credentials and then update the Users table with his FB id.

For the registration part, I luckily do not need anything else that FB can’t give me; so it’s a matter of pushing a button. The code is something like that:

$fbInfo = $fb->api_client->users_getInfo($fbId, array('name', 'pic_square', 'proxied_email'));
$model = new User();
$name = $fbInfo['name'];
$tries=1;
# find a unique username
do {
  $user = $model->fetchRow($model->select()->where('username=?', $name));
  if (!$user)
    break;
  else
    $name = $name . ($tries++);
} while(TRUE);
$user = $model->createRow();
$user->username = $name;
$user->avatar = $fbInfo['pic_square'];
$user->name = $fbInfo['name'];
$user->fb_proxy_email = $fbInfo['proxied_email'];
$user->fb_id = $fbId;
...
$user->save();
...

Finally, let me complain a bit: I find writing code that interacts with Facebook a very frustrating experience. The information you need is scattered all over the web (and the Facebook site itself, even), and examples quality is very poor. The technology behing FB is very sophisticated, though, and no matter what they try to do, you need to grasp some concepts that are sometimes quite difficult to master or a nightmare to debug.

Good luck.

Further readings

• The official Facebook Connect page (and the developers Wiki one)
• How Connect Authentication Works
• 10 Great Implementations of Facebook Connect
• What is behind Cross Domain Communication and even more
• Facebook API documentation

The plugin works adding a “special” unicode character after the given number of characters. By default U+200B (the zero width space) is used, but you can specify any other character as the second parameter. Only TEXT nodes are edited (the actual HTML tags and attributes remain untouched).

The plugin must be considered “experimental”, and should be better throughly tested if used in a production environment. Browsers, though, seem very “smart” about those Unicode “spaces” in that they do not mind them other then in a presentation context; in fact, I was in trouble thinking about HTML named entities… well, browsers don’t mind if a \U+200B is present in the middle of a &agrave;

Go to the breakly example page or just download it (a little help is present as the plugin comment).

Should you find a bug, please feel free to contact me.