Kus’ Blog

As you might have seen, there has been a long time since a fresh post, let’s change right now!

First of all, timestamp is correct… I don’t feel well this days and couldn’t sleep much… there’s nothing interesting on tv so I’ve been tweaking my blog and post the changes.

Until today I didn’t had an antispam solution for my comments and every couple of days I logged in and marked the spam as needed. What I’ve done now is installing a plugin called WP Captcha Free. It seems promising. It uses an ajax call to the server to get a hash token to make the post, since most bots doesn’t understand javascript, they try to comment directly and if no token is provided the comment is considered spam. I’ll come back and tell you if it works

I also have deactivated the “AddThis” plugin for the sharing icons and so, because most of you don’t even share at all or may be you do it through Menéame. So no chance for me to add the ones I want.

I really want to have those little icons to social-bookmark this, they are cool! I’ve found a way! Yeah, you are right another plugin, this its call obsocialbookmarker. At this point I wanted to add my own social bookmarks site to the plugin so I can handle all of them through it. La Tafanera, Fresqui and Menéame were missing so I have added them. On the next post I’ll explain all my tweaks!

Finally, I thought it would be nice to route my feeds through feedburner. I now most of you are feed readers, and I wanted to know how many of you are out there! Another plugin to do the job, recommended by the startup guide from feedburner.

Because evolution happens…

New job, new problems, new hacks! The next hack tries to take all the juice from linux’s pipes.

The task which I had to do was a database migration, from one mysql server to another one. The first problem were different versions, so hot copy wasn’t an option (actually few times is an option, unless it’s not in production ;)). The next big issue, were the size of the data, its big, really big (from my point of view) some DB reach 800M, for me is big enough to be a big problem

Mysqldump + mysql but this is good for 2 DB? I wanted to be selective in which DB to migrate, plus data should travel encrypted and transfer should delay as minimum as possible.

The script I got is this one (showing main loop, what’s interesting):

for db in `cat sm2-hf-dbs-kk.txt`
do
echo "Creant database a ${remote_sql}..."
ssh $remote_sql "echo \"create database if not exists ${db}\" | mysql -u ${db_user_rw} -p${db_pass_rw}"
echo "Fet"
echo "Donant permisos sobre la BD a rw..."
ssh $remote_sql "echo \"grant select, insert, update, lock tables, create, drop, alter on *.* to '${db_user_rw}'@'localhost';flush privileges\" | mysql -u ${remote_admin} -p${remote_admin_pass}"
echo "Fet"
echo "Fent dump gzipat i enviant a ${remote_sql}"
mysqldump --single-transaction -u ${db_user_r} -p${db_pass_r} ${db} | gzip | ssh $remote_sql "gunzip - | mysql -u ${db_user_rw} -p${db_pass_rw} ${db}"
echo "Fet"
done

The new server was empty, so we first ssh to it and from there we create a database. The next step was to give perms over the new db… I see now it can be done in one single line.

After this comes the magic. The script makes a dump of the db, it passes through gzip, this way we dramatically improve speed over the net. This pipe stream is passed through ssh to the other server unzip the info and put into the new mysql server. Pipe Magic rules!

I prefer  ssh the machine rather than setting a tunnel because the tunnel should set before the script, this way we don’t care. The other point is that the ssh should have a valid pair of keys to access it.

That’s all folks! Hope this enlightens someone…

Crec que era merescut que tinguessím el politó del rap del cuní per al mòbil, la veritat es que no el vaig buscar si no que me l’he tret jo del “videoclip” del youtube xD Polònia: El rap del Cuní, prou!, o és beatbox?

Una breu explicació del procés; vaig baixar el flv què és el video en flash amb el firefox i l’extensió “DownloadHelper”. Quan tenim el flv al disc usem la comanda

# ffmpeg -i fitxer.flv

que ens dona el següent resultat:

...
Input #0, flv, from 'Pol_nia__El_rap_del_Cun___prou___o__s_beatbox_.flv':
Duration: 00:01:04.9, start: 0.000000, bitrate: 56 kb/s
Stream #0.0: Video: flv, yuv420p, 320x240, 25.00 fps(r)
Stream #0.1: Audio: mp3, 22050 Hz, mono, 56 kb/s

Com veiem hi ha 2 streams, el de audio i el de video. Bé doncs, en teoria amb la comanda següent:

# ffmpeg -i fitxer.flv fitxer_audio.mp3

Podem obtenir el audio del fitxer i de fet així és, es pot reproduir, però no en el meu cas, que el meu mòbil no el pot reproduir… s’ha de recodificar ja que el fitxer que treu està en mpeg layer 2, o sigui mp2…

# file fitxer_audio.mp3
fitxer_audio.mp3: MPEG ADTS, layer II, v2, 64 kBits, 22.05 kHz, Monaural

Al ser mp2 ja no li mola al mòbil, així doncs l’hem de recuperar d’alguna altra manera. I aquí arriba mplayer, sisi amb el reproductor, ja us he dit que era potent.

Tal com indica el ffmpeg tenim l’audio codificat en mp3 així doncs amb el mplayer podem fer un dumpaudio que extreu la pista d’audio de un fitxer de vídeo.

# mplayer -dumpaudio fitxervideo.flv -dumpfile fitxeraudio.mp3

I això és tot! Ja tenim un fitxer mp3 extret de un vídeo del youtube!

De moment jo us deixo aquí un parell de fitxers que ja he extret:

- Rap del Cuní “ProuProuProu”

- “No Pas! Jo Votaré No Pas!” de Lluís Llac del Polònia és clar. Video Jo votaré no pas

Que us vagi de gust!

As you may have seen there have been lots of changes on the appereance fo this blog. Well I just wanted to make post to thanks to people that make this change happen:

• Smilies are borrowed from VikiWorks they are png that look quite good xD If you follow the instructions to change from gif to png take note that the file that is have to modify is wp-includes/functions.php
• I’ll integrate some type of minipost to quick blog from my twitter thanks to RudeWorks to show me the way to go.

I’ll keep on changing!

I must confess, I got into troubles while upgrading the installation of this wordpress.

Seems impossible how a such easy and user-friendly piece of code as wp is can go wrong and mess a little, just a little. Here it comes my little pain in the ass…

In this this story I’m much responsible as wp upgrade thing. When I created this hosting account and created a new mysql user for my blog I gave it limited grants to only select, insert, update and delete. When I got the upgrade of course it tells me that it has to upgrade my database, sure why not, help yourself wp! And here they are! lots of error of access denied while trying to alter or create tables in the database…

1st Round: Result, blog showing sql errors…

I quickly give my user the proper perms to let it make anything, then go to upgrade one more time and… what? Database is up-to-date… Uhm… You did it wrong wp…

A little ‘vim’ here and there and a solution was found. It seems that the upgrade.php script in wp don’t detects well access denied errors type and it updates the db_version option, that is located at wp_option table of your wp database. So I had to manually “downgrade” its value to let upgrade script rerun again.

Now everything is ok and posting.

While researching why the servers where overloaded I saw that some webpages where attacked/scanned from hundreds of different ips. Most of the where from Caravan Networks from Russia…

I know almost for sure that no webs pages in our servers have russian clients target so I wanted to ban them all, from their IP range. The problem was this range wasn’t trivial to get, so I did this little hack to get them.

Saddly it isn’t 100% effective but nearly 80%

First thing to get is a list of the attackers ip. In this case we got them from grepping through an apache access log.

cat /usr/local/apache/domlogs/kedume9/somedomain.com | grep "POST /web/ht" | cut -f 1 -d " " | sort -u malos.txt

did the trick.

Now the nice bash thing:

#!/bin/bash
for malo in `cat malos.txt`;
do
whois $malo | grep inetnum | awk '{print $2,$3,$4}' - | xargs ipcalc | grep '/' >> ranges.tmp
done
sort -u ranges.tmp > ranges.txt

As you see we are grepping inetnum from the whois as almost all ips showed the same information. This is a point of failure as it’s really possible that an ip range would have been marked in another fashion, as NetRange:. You will have to manually look for error in ipcalc.

Anyway, from a list of 2531 unique ips I got 33 internet ranges. Would be nice to check the origin country at this point.

Hope this helps to anyone

Last week I upgraded my work computer to the new Feisty. Of course such an upgrade installed a new kernel and it wasn’t supported by vmware…

Today I’ve got time and I found this page (VMWare and Ubuntu Feisty) a comenteer says to apply a patch to vmware, I did it and the modules compiled well.

Just to let you know

Today I was willing to listen to my music, but I have it on my iPod, the last.fm player didn’t worked and Rythmbox just sucks from my point of view. I found something new that seems its rocking the music players. The Banshee project Music Management and Playback for Gnome.

Right now I’m installing it I haven’t test it but they sell it well in their site. iPod - mp3 player integration, smart lists, last.fm plugin, internet radio and of course listen to your own music, sounds quite good.

For the installation I have follow the instructions from their Ubuntu Installation Guide just using deb http://directhex.mfgames.com/ ./ as the repository because the one they say have moved to it.

I’ll tell my experience with it later.

Bé, en la meva llaaaaaaaaaarga abscència durant alguns mesos tenia 33 comentaris per moderar, un dels quals no era spam!

Com tots bé sabem això de l’spam s’està convertint en una lacra per inet, quina ràbia, putus mafiosos.

En Lasker fa uns dies hem va comentar que el tema greylisting per a servidors de correu va força bé. El tema funciona de la següent manera: un sevidor que segueixi els estàndars envia un correu i si el server receptor el rebutja temporalment l’envia més tard; ara bé els que no segueixen cap tipus d’estàndard (llegeixis spammers) no esperen per tornar enviar el mateix missatge sino que passen i envien el correu a la següent víctima, perquè el que els interessa es enviar-ne el màxim possible al major nombre de gent possible.

En aquest cas, el greylisting envia un d’aquests missatges de rebuig temporal i apunta l’enviament al greylist, si es torna a rebre el correu més endevant en un cert interval de temps es deixa passar aquest correu.

He detectat últimament que es reben correus per duplicat i fins i tot triplicat, no sé si serà una estratègia per evitar el greylisting.

Aquí teniu una bona explicació sobre el greylisting. I pels usuaris de Postfix, és molt senzill posar-ho en pràctica, doneu un cop d’ull a Postgrey.

Caçadors d’IPs

Ahir vam poder caçar el(s) lladre(s) d’IPs del departament. Bé resulta que el sysadmin que hi havia abans que jo va donar una IP amb un doctorands per fer probes, però resulta que va donar la ip equivocada, la que va donar estava ocupada. Com que a la IP del punt d’accés (la ocupada) no s’hi accedia molt i els altres no l’usaven tampoc molt doncs mai, o quasi mai, es trobaven.

Per saber qui eren, bé la técnica del “ja es queixeran” no va acabar de funcionar com s’esperava, el que van fer va ser cambiar de roseta!! sort que la que van escollir la teniem ben registrada.

Pipe time

Bé ara en un minutet i fent una bona succesió de pipes i comandes de unix he aconseguit treure dels fitxers de log de l’apache les ips que hi han accedit i la data i hora de l’últim accés.

# cat /var/log/apache2/smop.no-ip.org-access.log | cut -d \ -f 1,4 | sort -r | sort -u -t \ -k 1,1 | sed -e "s/\[/-\ /"

Bàsicament el que fa és pillar el fitxer de log (se li pot posar un grep despres per pillar el que ens interessa), tallem la linia pels espais (-d \ ) i pillem els camps 1 i 4 (-f 1,4), després en fem un sort en ordre invers perquè així el sort següent ens treu l’última accés, sino treu el primer, bé en el segon sort doncs això sense repeticions (-u) usant com a delimitador un espai (-t \ ) i només tenint en compte la clau numbru 1 (-k 1,1). Per últim el tema del sed és més que res per fer la sortida un pèl més maca, sense lu del sed la data surt amb un “[” al davant, amb s/\[/-\ / ho substituïm per “- “. Estètica, com tot en aquest món, oi?

Spam

Avui he moderat el meu primer comentari, bé!! … era spam :(  aviam si algú s’anima a dir-hi la seva de debó

Rallamenta…

Com el cap de setmana passat, no bé de més rallant-se fent temes de la uni, presentació de SODX “Ubicació i selecció de servidors”, un tostón, sort que el dilluns ja ho acabem i fins el 30 no hi ha examen! Un de sol, si xD.