Schemathings

I’ll let you know!

Being more or less ‘from Maryland’ I am always on the lookout for a good crabcake. I was a busboy briefly at Thompson’s Sea Girt House when I was a teenager, which at the time had, according to Esquire Magazine, the World’s Best Crabcake. Mr. Thompson passed away quite some time ago so he might not mind me recommending Carr’s Whole Wheat Crackers with tuna salad with a touch of curry as a damn good fake crabcake

I’ve started blogging over at Not an MBA, a collaboration among several writers interested in current business administration and entrepreneurial topics.

I ran across the Template::Extract module while reading Spidering Hacks on the O’Reilly Hacks site and realized that I may have found the tool I need to migrate sites from Dreamweaver sites to a CMS backed system such as OpenACS (in as lazy a way as possible). What’s needed is really simple – just need to “balance the tags”.

With zero ado, here’s the simple test case I wrote that works quite nicely. Hope it forms a basis for a good project for someone else because none of the other samples I found on the web worked out of the box.

#!/usr/bin/perl -T
use Template::Extract;
use Data::Dumper;
use LWP::Simple qw(get);

my $obj = Template::Extract->new;

my $template = < < '.';
[% ... %]<!-- #BeginTemplate "[% Template %]" -->[% ... %]
[% FOREACH editable %]
[% ... %]
<!– #BeginEditable “[% regionname %]” –>
[% ... %]
<!– #EndEditable –>
[% END %]
[% ... %]
<!– #EndTemplate –>
[% ... %]
.

# – Network version
my $document = get “http://www.cmbs.org/career_main.htm” or die “Can’t get page”;

# – Filesystem version
#open INPUT, “;
#close INPUT;

#print Dumper @document;

print Data::Dumper::Dumper( $obj->extract($template, $document));

Here is some example output:

$VAR1 = {
‘editable’ => [
{
'regionname' => 'doctitle'
},
{
'regionname' => 'topnav'
},
{
'regionname' => 'subtitle'
},
{
'regionname' => 'body'
}
],
‘Template’ => ‘/Templates/final.dwt’
};

Just finished reading I got from my Dad for Christmas. I suppose a lot of people read up on Afghani/Pakistani culture immediately after 9/11 to gain some understanding of what might drive people to commit such an attack on America. I decided to read to gain some historical perspective on the region as well as my normal ‘hot spots’ resource – . I believe there is a TV show based on the latter book at this point, not sure – I don’t get to watch much TV.

So The Warlord’s Son was a nice refresher on the region – the story is about a print journalist who flies to Peshawar to cover the region immediately after 9/11 – I’m not into spoilers, but I found the story to be well informed and very engaging as a thriller.

Most interesting to me was the anthropological angle of Pashtun culture – the Pashtunwali code – malmastiya and nanataway – the obligation to provide safe harbor to visitors who come in peace. I met a 7-11 owner a few months after 9/11 – he was Pashtun (there are no nationalities if you are Pashtun !) and I was very impressed by his straightforwardness and clarity about world politics – it was a refreshing meeting during a period of intense bombast on our TV. We spoke for several hours about the region he came from, the history of other countries fighting over Khyber Pass to guarantee trade access through the mountains between whatever-istan it happens to be in the current century.

Anyway, I’m rambling – read the book – it’s good!

An important step in creating a logical standby of your Oracle 10G database when using dataguard is to ensure that all the data is compatible.

These two queries will help you quickly spot what you need:

select distinct owner, table_name from dba_logstdby_unsupported order by owner, table_name;

select owner, table_name, bad_column from dba_logstdby_not_unique where table_name not in (select table_name from dba_logstdby_unsupported);


Well maybe not exactly magic, but pretty nice nonetheless. In the previous entry I discussed managing the standby archivelogs (in the flashback recovery area). If you are running everything with a catalog database and have a primary and a standby, the primary and standby are pretty much identical except in the flashback area:

/path/to/flashback/archivelog/$DB_UNIQUE_NAME/data/files

so on the primary and standby, which have different $DB_UNIQUE_NAME, there is a directory unique to each machine. If you
ever need to clear out files on one or the other and don’t use RMAN to do it, this will make sure the catalog is accurate:

- use rman to connect to the primary and catalog
- RMAN> crosscheck archivelog all like ‘%$DB_UNIQUE_NAME%’;
Do the same on the standby…

This only updates the catalog for the online archivelog files. To update the backup files as well:

run {
allocate channel device type disk;
crosscheck backup;
}

Turns out to be quite easy actually. I do backups from the primary database to a local drive and have been puzzled how to delete the standby archive logs after they ship. The ‘obvious’ solution is the documented feature in RMAN:

CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY

However, that didn’t work when I originally setup RMAN. Turns out that because I’m using ‘Maximum Performance Mode’, my standby archivelog destination is NOT mandatory and therefore Oracle only half pays attention to it.

Finally (August 2005) Oracle released a Metalink Note 331924.1 explaining how to enable this feature of RMAN with either Max Performance or Max Availability mode – just add _log_deletion_policy=’ALL’ to the spfile, restart your database and all should be good (haven’t tested yet – have to ask for reboot time).

Anyway – back to those archivelogs on the standby.

Turns out that RMAN, when you are using a catalog, IS aware of the standby logs, but just can’t delete them if you connected to the primary database. If you invoke RMAN from the standby machine, you can just use regular RMAN commands to manage the files. I had been just deleting archivelogs manually from the drive on the standby once they were old, to keep room available – turns out eventually (9 months on a fairly busy server) I got this message in the alert log, that finally forced me to do it right (although I will take some comfort in getting it wrong since Oracle documented the feature a year after releasing the software):

ORA-19815: WARNING: db_recovery_file_dest_size of 62914560000 bytes is 95.59% used, and has 2773004800 remaining bytes available.

What’s strange and let me know that something was horribly wrong, is that there were only 4Gb of files in that area, and the message says that 95.9% of the 60 Gb is used.

Anyway, do what I said above – use RMAN to connect to your standby d/b and the catalog, and for example to delete up to 5 days ago of archivelogs sitting in the flashback area:

run {
allocate channel for maintenance device type disk;
delete archivelog until time 'sysdate -5';
}

If you create these files with a directory path specification, you end up with empty directories as each older day's archivelogs get cleaned out, so eventually you will have to hand delete these.

My friend Talli pointed out some interesting links right as I am in the middle of reconsidering the oracle/mysql/postgresql – which is better debate. With the newly released mysql5.0 and oracle’s investment in their innodb technology, it looks like a killer platform, but postgresql 8.0 has some killer features and I’ve always liked the more ‘academically’ correct approach to the relational model that postgresql has.

Feed lounge has switched from mysql to postgresql with good results.

Sun is backing postgresql as well.

Guess I need to rethink taking the road more travelled (mysql) once again ..

One of the databases I support is an Oracle 10G instance on Windows 2003 Server running in a physical standby configuration. One thing I still need to tweak is how archive logs get deleted on the standby machine. I can’t see how to get RMAN to address the archivelogs on the other machine. Well, the other day the archivelog destination drive filled up on the standby, so the primary stopped shipping logfiles. The sysadmin noticed this and cleared out some older logfiles, but we had an archivelog gap – the archivelogs that were older than the retention period on the primary were no longer online in the flashback recovery area.

Fortunately we keep disk-based RMAN backups online as long as possible so I knew they were still embedded in the backupset somewhere.

Here’s the secret recipe for getting them back out:
Look at your OEM and figure out which logfile sequence numbers are in the gap.
Login to your RMAN and do this:
RMAN> list backupset;

— snip snip —

List of Archived Logs in backup set 40885
Thrd Seq Low SCN Low Time Next SCN Next Time
—- ——- ———- ——— ———- ———
1 12870 132214214 31-OCT-05 132240966 31-OCT-05
1 12871 132240966 31-OCT-05 132291702 01-NOV-05
1 12872 132291702 01-NOV-05 132400078 01-NOV-05
1 12873 132400078 01-NOV-05 132504581 01-NOV-05
1 12874 132504581 01-NOV-05 132594119 01-NOV-05
1 12875 132594119 01-NOV-05 132683505 01-NOV-05

— snip snip —

I am just showing a few of the Sequence numbers – the output is quite long.
Let’s say I needed 12872-1874.

RMAN> run {
2> ALLOCATE CHANNEL c1 DEVICE TYPE DISK;
3> restore archivelog sequence 12872;
4> restore archivelog sequence 12873;
5> restore archivelog sequence 12874;
}

You’ll see something like this:
allocated channel: c1
channel c1: sid=2054 devtype=DISK

Starting restore at 04-NOV-05

channel c1: starting archive log restore to default destination
channel c1: restoring archive log
archive log thread=1 sequence=12872
channel c1: restored backup piece 1
piece handle=G:\BACKUP\ARCHIVE\ARC_DORAMYSID_T573243355_S1283_P1 tag=TAG20051101T181555
channel c1: restore complete
Finished restore at 04-NOV-05

Repeated for each recovered sequence …

What’s really slick is the second they are recovered, the primary will ship the archivelogs and the standby will apply them. Recovery from being 3-4 days behind on a very busy instance only took about 10 minutes at which point the standby was fully in sync again.

If they weren’t in the online backups, I would have had to look in RMAN to figure out which tapeset they were in, restore the backups, and run a CROSSCHECK to let Oracle know they were back online.

Someday I’ll configure RMAN to use the tape farm directly.