Editing Incubator escapee wiki:Help desk/Archive 2 (section)

== SQL/Compression problem with wikipedia-files (old, read 10GB-files) ==

Hi,

I couldn't find a forum to ask for help other than this one (although I am aware that it is somewhat off-topic, sorry, I am desperated after 2 days of unsuccessful trials right now).

I downloaded the english version of the whole wikipedia-history (some 10GB) and installed it in a MySQL-database. The data is present. I can access it by e.g. "mysql -> SELECT old_title FROM old;" or with a perl-script that opens the database via DBI, executes any SQL-script and get the results on my screen. On this side no problem.
What I can't access is the actual (uncompressed, human readable) content of the old_text-column itself. It's compressed, ok. I got that from the php-source of mediawiki. But what's wrong with the following code?

 my $sth = $dbh->prepare("SELECT old_title, old_text FROM old");
 $sth->execute;
 while (my @row = $sth->fetchrow_array ) {
  if(defined $row[0]) {
    print $row[0] . " ";
  }
  if(defined $row[1]) {
    my $a = uncompress($row[1]); # dummy variable 
    print $a . "\n";
  }
 }


$a is not defined after this call. If I implement another possibility of Compress::Zlib like:

 sub decompressText {
  my $d;
  my $status;
  my $out;
  my $out2;
  ($d, $status) = deflateInit(); #-Level => Z_BEST_COMPRESSION);
  $status == Z_OK
    or die "INIT failed\n" ;
  ($out, $status) = $d->deflate($_[0]) ;
  print "STATUS " . $status ."\n";
  $status == Z_OK
    or die "DEFLATE failed\n" ;
  ($out2, $status) = $d->flush() ;
  $status == Z_OK
    or die "FLUSH failed\n" ;
  if(defined $out) { print "DEFINED.\n";}
  if(defined $out2) { print "DEFINED2.\n";}
  my $z = $out . $out2;
  return $z;
 }

I just get decompressText(old_text) to be the binary-stuff I got with the simple SELECT-statement without any (de)compression at all. According to the perldoc both failures indicate that the uncompression wasn't successful. 

So my question is: how do I uncompress "old_text" in the table "old"????

Please, please help. I tried it for 2 days and I am pretty sure the error is obvious.

THANK YOU!

[[User:KaHa242|KaHa242]]


: Eek, too technical for us!  You probably want the mediawiki list: gmane.org.wikimedia.mediawiki [[Mailing_lists#Mediawiki]] -- [[User:Finlay McWalter|Finlay McWalter]] | [[User talk:Finlay McWalter | Talk]] 21:58, 8 Jun 2004 (UTC)