Terry’s Worklog

So I’ve just about finished making the changes to MarcEdit 5.0 to include Saxon’s .NET component into the application.  This will allow users the ability to support XSLT 1.0, 2.0, XPath, Xquery, etc.  By default, the application will continue to utilize the customized version of the System.XML classes provided by default in .NET since they are a bit faster than the Saxon implementation when dealing with 1.0 documents — but users will have the option to change this setting which might be the way to go depending on your XML.  The System.XML component is fairly strict and doesn’t support non-numeric character entities (i.e., &copy isn’t supported) while the Saxon component does. 

Anyway, the setting will be found in the properties box — so it will show up like:

I’ve also been updating a number of the XSLT files provided with MarcEdit, with the biggest updates to the Dublin Core translations (both OAI harvester and straight DC translation).  And with XSLT 2.0 support added to MarcEdit, I’ll likely provide LC’s FRBR code with MarcEdit shortly as well.

Lastly, I’ll be changing the MarcEdit license shortly.  I’d like to make it easier for people to use and develop with the program so I’m looking at something like an MIT-like license.  Anyway, I’m thinking of using something like:

Redistribution and use in binary forms are permitted provided that the following conditions are met: 

• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 

• Neither the name of TERRY REESE nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 Anyway, I want to do a bit more testing to make sure that the COM changes, XSLT changes, etc. all work and then I’ll post.

–TR

I was playing around with Saxon.NET and I like it.  Its also very well done (not suprisingly).  Benchmarking, I found that its competitive with my custom processor for XSLT 1.0 while providing 2.0 support.  I’ll be integrating this into MarcEdit, likely even this weekend, so folks can start using XSLT 2.0 in the application.  I just need to figure out exactly how I’ll implement this engine — partly because my custom processor still outperforms when using XSLT 1.0.  So I’ll likely mix the two processors — but allow users to define which that they want to use. 

Anyway, here’s a quick sample of how you utilize the XSLT processor in C#.

new System.IO.StringWriter();stringWriter.Write(reader.ReadToEnd());

System.IO.TextReader stringReader = new System.IO.StringReader(stringWriter.ToString());System.Xml.XmlTextReader reader2 = new System.Xml.XmlTextReader(stringReader);reader2.XmlResolver =

null;// Load the source document

XdmNode input = processor.NewDocumentBuilder().Build(reader2);

// Create a transformer for the stylesheet.

XsltTransformer transformer = processor.NewXsltCompiler().Compile(new System.Uri(@”c:\xslt\EADlitetoMARC21slimXML.xsl”)).Load();transformer.InputXmlResolver = null; 

// Set the root node of the source document to be the initial context node

transformer.InitialContextNode = input;

// Create a serializer

Serializer serializer = new Serializer();//serializer.SetOutputWriter(Console.Out);

serializer.SetOutputFile(@”c:\xmlstream.xml”);

// Transform the source XML to System.out.

transformer.Run(serializer);

This weekend, Kyle Banerjee and I drove down to Shasta for the Shasta Super Century.  This is a 135 mile, 16,500 ft vertical gain ride the finished with a quad-burning ride up Mt. Shasta.  This was a ride that I’d been looking forward too but apparently my body wasn’t.  About 6 miles into the first major climb (10 mile, 3800 ft. climb) my back knotted up.  I managed to finish up the hill and figured that maybe this would stretch itself out with a few ibprofin.  I took 4, costed down the hill and made my way to the next rest area.  After a quick water break, we started up the next major climb.  ~10 miles up, I just about fell off my bike as my body said no more.  My back was in knots — I could peddle no further.  I limped my way back to the finish line, finishing only 85-90 miles of the 135 mile course.  After an hour in the bed of the pickup, my back finally stop spasming enough for me to go change and wait for Kyle to finish the ride.  And finish he did.  He finished in ~13 hours and still standing.  I wish I could have rode in with him.

Anyway, this was a disappointing way to finish my big ride session.  I’ll do maybe two more centuries this year, but none of them that are this challenging.  As to why my body had so much trouble, I’m not sure.  The distance wasn’t a problem (I’ve ridden further) and the hills shouldn’t have been a problem (at least the first 3) as I’ve ridden equivalents.  The nearest I can tell is that either:

• I didn’t train well enough for this particular ride — which might have been the case.  I was really focused this year on training for the STP — and that ride turned out as good as I could have asked.  I was able to do it fast and easy.  But I really haven’t done many hill workouts and that might have bit me.
• Bike fitting…I’d actually lowered my bit handlebars the day of the ride in order to give myself a more agressive bike position.  I’ve done this before on flat courses – but I think that the extra torque that this put onto my back probably caused the problem.  I probably shouldn’t have made this change the day of.

Anyway, look like next year I’ll be going down to Shasta to finish this ride.  I’d never not finished a ride and I owe this mountain a climb.  So I guess I’ll be visiting Shasta again next year to give this another go.

–TR

I don’t do it very often but its now been about a year since I formally posted MarcEdit 5.0 as beta and in the 12 months, the program has now been downloaded by 10, 021 unique users.  Not bad I guess.  Hopefully, when I take the program to 5.1 when the docs are completed, I’ll see less 4.6 downloads (there were still 3000+ downloads over last year — though the number of downloads has been ~10 per week over the past 3-3 1/2 months), so I’m hoping folks are starting to find things to make the switch worth while. 

–TR

So today I entered the wonderful world of skype.  Love it.  If anyone has a desire to chat and use skype, my skype name is reese_terry.

–TR

I’m not sure how I missed this, but apparently the US Patent office, in their infininte wisdom, granted Blackboard a paton on “Internet-based education support system and methods”.    Michael Feldstein on the e-Literate blog notes that:

As I read it, Blackboard basically owns the patent on any sort of groupware at all that is used for teaching purposes. This could have very serious consequences for both proprietary and Open Source competitors–and I define “competitors” as loosely as possible. (You could probably slam Drupal with this under the right circumstances, for example.)

He has some additional comments on his post at: http://mfeldstein.com/index.php/weblog/permalink/blackboard_patents_the_lms and posts relevant snippets from the patent.  I would have missed this, but Blackboard has started trying to enforce its patent.  On July 26th, Blackboard issues a demand for a jury trial against Desire2Learn, Inc. for patent infringement (see here: http://www.theinquirer.net/images/articles/blackboard.pdf).  I can’t see how this patent holds up to the scrutiny of a jury trial — but I have a difficult time believing this yahoos got a patent for this idea in the first place.

–TR

I’ve posted an update.  Some of the highlights of the change:

• New functions added to the MarcEngine/COM objects that allow for the streaming of text into the MARCEngine for direct translation to MARC.  Traditionally, MarcEdit has been a file based tool (i.e., you send it a file and it prints the results to a new file).  The streaming functions allow for raw data to be dropped on the engine with raw data being returned.  The first of these such functions is a stream function that allows for the processing of MARCXML data.  Here’s an example of its use in a vbscript:

const CONVERT_TO_MARC = 1
const DEFAULT = 0
const CONVERT_TO_UTF8 = 2

Dim obj_MARC
Dim sMARC
Dim sXML

Set obj_MARC=CreateObject(”MARCEngine5.MARC21″)
Set fso = CreateObject(”scripting.FileSystemObject”)
Set myfile = fso.OpenTextFile(”marcxml.xml”,1, false)
sXML = myfile.ReadAll
myfile.close

sMARC = obj_MARC.ReadMARC21XMLStream(sXML, “C:\net_marcedit\C#\MProgram\MarcEdit\bin\Debug\XSLT\MARC21slim2Mnemonic.xsl”, false, CONVERT_TO_MARC)

set myfile = fso.OpenTextFile(”test_a.mrc”, 2, true)
myfile.write(sMARC)
myfile.close()
msgbox “finished”

• MarcEdit interface, I’ve exposed more of the character translation functions and removed a few of the self imposed barriers as to where these translations can be run.
• Lots of work while at OSCON on making changes to support MONO.  The gui still isn’t as clean as I’d like, but its running.
• Script Maker changes: I’ve readded the ability to edit scripts generated by the script maker.  In order for this to work, you need to generate scripts with this new script maker version.

As always, the tool can be downloaded from: MarcEdit50_Setup.exe.

–TR