Cleaning content from OpenOffice using Perl

Open office is great software for a number of things - I use it as my office software instead of paying a premium for Microsoft office. But one thing it's not so hot at is converting documents to clean HTML. And one of the main things I use it for is adding content to sites that clients send me in word files or excel spreadsheets.

Of course, you can always cut and paste, but that loses a lot of formatting. For example, if the content uses a lot of italics, bold text, etc. it can be a huge pain to go back and put all that back in.

Another common situation is a client sending some sort of tablular data in a spreadsheet - for example a list of events. It's the kind of data that can change a lot, and it also needs to be in a table with some decent formatting to be usable. Doing it manually is a lot of grunt work.

But grunt work is what computers excel at, and I'm not very good at. So I've developed a number of perl scripts to help streamline this kind of job. I'll go over one below - feel free to use it, modify it, etc.

Background and Preparation
First, this is coming from an excel spreadsheet that the user has sent to update their site. I first open it in OpenOffice, then delete in rows or colums that may needed to be deleted. For example, in this case there was a "Notes" column that shouldn't appear on the live site, so I delete that. Then I save it as HTML using OpenOffice's standard converter. I end up with an HTML file that includes tons of non-compliant HTML, such as unquoted values, uppercase tags, etc. - as well as some weird stuff that I assume OpenOffice uses somehow such as this: SDNUM="1033;0;[$-409]MMMM D\, YYYY;@". We're going to use a perl script to strip all that out and convert this mess to clean HTML.

The Perl Script
Below is my script. It may not be the most elegant or smallest number of characters, but I do think it's easy to read. It basically steps through each line of file and does a number of regular expressions to remove excess crap, add in the alternating row color classes, etc. It also will only output the actual table HTML, not the rest of the page html, like the head tags, etc.

The script is run on the command line and just prints out its results. That makes it easy to check the output, and if I need to save it to a file I can just use the ">" operator to redirect the output to a file, then pull that file into the page it's needed on.

my $out = "";
my $cntr = "0";
my $file = $ARGV[0];
my $intable = 0;

# open file and read through lines
open FILE, "$file" or die "Couldn't open file!\n";
while ($_ = ) {
        if ($_ !~ /\w/) { next } # skip empty

        if ($_ =~ /<table/i) { $intable = 1; } # we're inside the table

        # junk to remove!
        $_ =~ s/ (width|height)=(\d\d\d|\d\d)//ig;
        $_ =~ s/ align=(left|right)//ig;
        $_ =~ s/ (sdnum|sdval)=".*?"//ig;
        $_ =~ s/ bgcolor=".*?"//ig; # remove bgcolor on tds
        $_ =~ s/<font .*?>//ig; # remove font tags
        $_ =~ s/<\/font.*?>//ig;
        if ($_ =~ /<\/?([A-Z]*?)>/) { # this converts tags to lower case
                $l = lc $1;
                $_ =~ s/$1/$l/g;
                }

        $_ =~ s/<td><br><\/td>/<td> <\/td>/ig;
        $_ =~ s/<u><a href/<a href/ig;
        $_ =~ s/<\/a><\/u>/<\/a>/ig;


        if ($_ =~ /<tr/) { # this adds in alternating row color classes
                $rc = "class=\"rowcolor" . $cntr % 2 . "\"";
                $_ =~ s/<tr/<tr $rc/g;
                $cntr++;
                }

        if ($intable) { $out .= $_; } # we're in the table, so save the line for output

        if ($_ =~ /<\/table/i) { $intable = 0; }
        }

print $out;
# all done