RSS 2.0
Sign In
# Thursday, May 15, 2008

Many times I was tempted to create my bests books list.

No, I dare not to create it. There are so many brilliant masterpieces; I allow to make such lists to someone, who is wiser than myself. However, I cannot be silent about a book I've read recently. This is Ulysses by James Joyce.

It's hard reading: you need to know all creative work of Joyce, Homer, state of the affairs in the world in the year of 1904, map of city of Dublin in that year, and many many other information. That is the reason why you must read annotated version.

The labour of reading is compensated with intellectual pleasure from it, because it's not a story that is catching you, but narration. Having imagination, you're immersing into a world created by the author.

I can believe that there are people for years living in the world created by Joyce.

Thursday, May 15, 2008 6:22:57 AM UTC  #    Comments [0] -

# Monday, May 12, 2008

Today I've found another new language (working draft in fact). It's an XML Pipeline Language.

XProc: An XML Pipeline Language, a language for describing operations to be performed on XML documents.

An XML Pipeline specifies a sequence of operations to be performed on zero or more XML documents. Pipelines generally accept zero or more XML documents as input and produce zero or more XML documents as output. Pipelines are made up of simple steps which perform atomic operations on XML documents and constructs similar to conditionals, iteration, and exception handlers which control which steps are executed.

An experience shows a process of language invention is an essential part of computer industry from the very beginning, however...

I must confess I must be too reluctant to any new language: I was happy with C++, but then all these new languages like Delphi, Java, C#, and so many others started to appear. It's correct to say that there is no efficient universal language, however I think it's wrong to say that a domain specific language is required to solve a particular problem in a most efficient way.

And now a question to the point: why do you need a new language for describing operations to be performed on XML documents?

Monday, May 12, 2008 7:11:58 AM UTC  #    Comments [0] -
xslt
# Friday, May 9, 2008
Георгиевская ленточка

A project I'm currently working on, requires me to manipulate with a big number of documents. This includes accessing these documents with key() function.

I never thought this task poses any problem, until I've discovered that Saxon caches documents loaded using document() function to preserve their identities:

By default, this function is ·stable·. Two calls on this function return the same document node if the same URI Reference (after resolution to an absolute URI Reference) is supplied to both calls. Thus, the following expression (if it does not raise an error) will always be true:

doc("foo.xml") is doc("foo.xml")

However, for performance reasons, implementations may provide a user option to evaluate the function without a guarantee of stability. The manner in which any such option is provided is implementation-defined. If the user has not selected such an option, a call of the function must either return a stable result or must raise an error: [err:FODC0003].

Saxon provides a saxon:discard-document() function to release documents from cache. The use case is like this:

<xsl:variable name="document" as="document-node()"
   select="saxon:discard-document(document(...))"/>

You may see, that saxon:discard-document() is bound to a place where document is loaded. In my case this is inefficient, as my code repeatedly accesses documents from different places. To release loaded documents I need to collect them after main processing.

Other issue in Saxon is that, processor may keep document references through xsl:key, thus saxon:discard-document() provides no guaranty of documents to be garbage collected.

To deal with this, I've designed (Saxon specific) api to manage document pools:

t:begin-document-pool-scope() as item()
  Begins document pool scope.
    Returns scope id.

t:end-document-pool-scope(scope as item())
  Terminates document pool scope.
    $scope - scope id.

t:put-document-in-pool(document as document-node()) as document-node()
  Puts a document into a current scope of document pool.
    $document - a document to put into the document pool.
    Returns the same document node.

The use case is:

<xsl:variable name="scope" select="t:begin-document-pool-scope()"/>

<xsl:sequence select="t:assert($scope)"/>

...
<xsl:variable name="document" as="document-node()"
  select="t:put-document-in-pool(...)"/>
...

<xsl:sequence select="t:end-document-pool-scope($scope)"/>

Download document-pool.xslt to use this api.

Friday, May 9, 2008 6:58:29 AM UTC  #    Comments [0] -
xslt
# Saturday, May 3, 2008

I was already writing about the logical difference between tamplates and functions. This time I've realized another, technical one. It's related to lazy evaluation, permitted by language specification.

I was arguing as follows:

  • suppose you define a function returning a sequence;
  • this function at final step constructs document using xsl:result-document;
  • caller invokes this function and uses only first item of sequence;
  • lazy evaluation allows to xslt processor to calculate first item only, thus to avoid creation of output document altogether.

This conclusion looked ridiculous to me, as it means that I cannot reliably expect creation of documents built with xsl:result-document instruction.

To resolve the issue I've checked specification. Someone has already thought of this. This is what specification says:

[Definition: Each instruction in the stylesheet is evaluated in one of two possible output states: final output state or temporary output state].

[Definition: The first of the two output states is called final output state. This state applies when instructions are writing to a final result tree.]

[Definition: The second of the two output states is called temporary output state. This state applies when instructions are writing to a temporary tree or any other non-final destination.]

The instructions in the initial template are evaluated in final output state. An instruction is evaluated in the same output state as its calling instruction, except that xsl:variable, xsl:param, xsl:with-param, xsl:attribute, xsl:comment, xsl:processing-instruction, xsl:namespace, xsl:value-of, xsl:function, xsl:key, xsl:sort, and xsl:message always evaluate the instructions in their contained sequence constructor in temporary output state.

[ERR XTDE1480] It is a non-recoverable dynamic error to evaluate the xsl:result-document instruction in temporary output state.

As you can see, xsl:function is always evaluated in temporary output state, and cannot contain xsl:result-document, in contrast to xsl:template, which may be evaluated in final output state. This difference dictates the role of templates as a "top level functions" and functions as standalone algorithms.

You can find more on subject at "Lazy evaluation and predicted results".

Saturday, May 3, 2008 4:36:38 PM UTC  #    Comments [0] -
xslt
# Wednesday, April 30, 2008

In the era of parallel processing it's so natural to inscribe your favorite programming language in the league of "Multithreading supporter". I've seen such appeals before "Wide Finder in XSLT --> deriving new requirements for efficiency in XSLT processors."

... I am not aware of any XSLT implementation that provides explicit or implicit support for parallel processing (with the obvious goal to take advantage of the multi-core processors that have almost reached a "prevalent" status today) ...

I think both xslt and xquery are well fitted for parrallel processing in terms of type system. This is because of "immutable" nature (until recent additions) of the execution state, which prevents many race conditions. The only missing ingredients are indirect function call, and a couple of core functions to queue parallel tasks.

Suppose there is a type to encapsulate a function call (say function-id), and a function accepting a sequence and a function-id. This function calls function-id for each element of the sequence in a parallel way, and then combines a final result, as if it were implemented serially.

Pretty simple, isn't it?

<!--
  This function runs $id function for each item in a sequence.
    $items - items to process.
    $id - function id.
    Returns a sequece of results of calls to $id function.
-->
<xsl:function name="x:queue-tasks" as="items()*">
  <xsl:param name="items" as="item()*"/>
  <xsl:param name="id" as="x:function-id"/>

  <!-- The pseudo code. -->
  <xsl:sequence select="$items/call $id (.)"/>
</xsl:function>

Wednesday, April 30, 2008 6:59:29 AM UTC  #    Comments [0] -
xslt
# Sunday, April 27, 2008

For the last several weeks I was on my military duty. We were patrolling Israeli border near the Egypt. It was a completely different world, world of guns, Hummers, heat sensors...

There I've met my army friends. It was fun to listen stories they were telling. At some point I've started to realize that I'm growing older. Most of my friends are married and have two or three children.

It seems, this was the genuine world, and my own one is fictitious.

Sunday, April 27, 2008 2:54:39 PM UTC  #    Comments [0] -

# Saturday, April 5, 2008

Does WebSphere MQ library for .NET support a connection pool? This is the question, which ask many .NET developers who deal with IBM WebSphere MQ and write multithread applications. The answer to this question unfortunately is NO… The .NET version supports only individual connection types.

I have compared two MQ libraries Java's and one for .NET, and I’ve found that most of the classes have the same declarations except one crucial for me difference. As opposed to .NET, the Java MQ library provides several classes implementing MQ connection pooling. There is nothing similar in .NET library.

There are few common workarounds for this annoying restriction. One of such workarounds (is recommended by IBM in their “MQ using .NET”) is to keep open one MQ connection per thread. Unfortunately such approach is not working for ASP.NET applications (including web services).

The good news is that starting from service pack 5 for MQ 5.3, and of course for MQ 6.xx they are supporting sharing MQ connections in blocked mode:

“The implementation of WebSphere MQ .NET ensures that, for a given connection (MQQueueManager object instance), all access to the target WebSphere MQ queue manager is synchronized. The default behavior is that a thread that wants to issue a call to a queue manager is blocked until all other calls in progress for that connection are complete.”

This allows creating an MQ connection (pay attention that MQQueueManager object is a wrapper for MQ connection) in one thread and exclusive use it in another thread without side-effects caused by multithreading.

Taking in account this feature, I’ve created a simple MQ connection pool. It’s ease in use. The main class MQPoolManager has only two static methods:

public static MQQueueManager Get(string QueueManagerName, string ChannelName, string ConnectionName);

and

public static void Release(ref MQQueueManager queueManager);

The method Get returns MQ queue manager (either existing from pool or newly created one), and Release returns it to the connection pool. Internally the logic of MQPoolManager tracks expired connections and do some finalizations, if need.

So, you may use one MQ connection pool per application domain without additional efforts and big changes in existing applications.

By the way, this approach has allowed us to optimize performance of MQ part considerably in one of ours projects.

Later on...

To clarify using of MQPoolManager I've decided to show here following code snippet:

  MQQueueManager queueManager = 
MQPoolManager.Get(QueueManagerName, ChannelName, ConnectionName); try { // TODO: some work with MQ here } finally { MQPoolManager.Release(ref queueManager); } // at this point the queueManager is null

Saturday, April 5, 2008 8:55:57 PM UTC  #    Comments [7] -
Tips and tricks

Yesterday's idea has inspired me as much as to create a prototype implementation of map and tuple in the xslt 2.0.

Definitely I wished these were a built-in types, and were considered as atomic values for purposes of comparasions and iteration. This way it were possible to create highly efficient grouping per several fields at once.

This pure implementation (xslt-tuple.zip) is rather scetchy, however it allows to feel what can be done with tuples and maps. I guess a good example may say more than many other words, so have a pleasure:

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:f="http://www.nesterovsky-bros.com/xslt/functions"
  exclude-result-prefixes="xs f">

  <xsl:include href="tuple.xslt"/>
  <xsl:include href="map.xslt"/>

  <xsl:template match="/">
    <root>
      <xsl:variable name="tuples" as="item()*" select="
        f:tuple
        (
          for $i in 1 to 10 return
            f:tuple(1 to $i)
        )"/>

      <total-items>
        <xsl:sequence select="count($tuples)"/>
      </total-items>
      <tuples-size>
        <xsl:sequence select="f:tuple-size($tuples)"/>
      </tuples-size>
      <sums-per-tuples>
        <xsl:for-each select="1 to f:tuple-size($tuples)">
          <xsl:variable name="index" as="xs:integer" select="position()"/>

          <sum
            index="{$index}"
            value="{sum(f:tuple-items(f:tuple-item($tuples, $index)))}"/>
        </xsl:for-each>
      </sums-per-tuples>

      <xsl:variable name="cities" as="element()*">
        <city name="Jerusalem" country="Israel"/>
        <city name="London" country="Great Britain"/>
        <city name="Paris" country="France"/>
        <city name="New York" country="USA"/>
        <city name="Moscow" country="Russia"/>
        <city name="Tel Aviv" country="Israel"/>
        <city name="St. Petersburg" country="Russia"/>
      </xsl:variable>

      <xsl:variable name="map" as="item()*" select="
        f:map
        (
          for $city in $cities return
            ($city/string(@country), $city)
        )"/>

      <xsl:for-each select="f:map-keys($map)">
        <xsl:variable name="key" as="xs:string" select="."/>

        <country name="{$key}">
          <xsl:sequence select="f:map-value($map, $key)"/>
        </country>
      </xsl:for-each>
    </root>
  </xsl:template>

</xsl:stylesheet>

Saturday, April 5, 2008 3:49:03 PM UTC  #    Comments [0] -
xslt
# Friday, April 4, 2008

The type system of xslt 2.0 is not complete (see Sequence of sequences in xslt 2.0). You cannot perform manipulations over items as you could do. The reason is in the luck of set based constructs: xslt 2.0 supports sequences, but not associative maps of items.

If you think that xml can be used as a good approximation of a map, I shan't agree with you. Xml has an application in a very specific cases only. Maps I'm thinking of,  would allow associate items by reference, like sequences do.

This opens a perspective to create a state objects, to manage sequence of sequences, to create cyclic graphs of items, and so on. These maps are richer than what key() function provides right now, and allow to implement for-each-group in xquery.

Such maps can be modeled with several functions, however I would wish they were built in:

f:map($items as item()*) as item()
Returns a map from a sequence $items of pairs (key, value).

f:map-items($map as item()) as item()*
Returns a sequence of pairs (key, value) for a map $map.

f:map-keys($map as item()) as item()*
Returns a sequence of keys contained in a map $map.

f:map-values($map as item()) as item()*
Returns a sequence of values contained in a map $map.

f:map-value($map as item(), $key as item()) as item()*
Returns a sequence of values corresponding to a specified key $key contained a specified map $map.

The other thing I would add is items tuple. It's like a sequence, however a sequence of tuples is never transformed into single sequence, but stays as sequence of tuples.

Fortunately it's possible to implement such extension functions.

Friday, April 4, 2008 1:49:56 PM UTC  #    Comments [0] -
xslt
# Wednesday, April 2, 2008

xslt 2.0 is a beautiful language and at the same time it allows constructs, which may trouble anyone.

Look at this valid stylesheet:

<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xsl:template match="/">
    <xsl:variable name="x" as="node()" select="."/>
    <xsl:variable name="x" as="xs:int" select="***"/>

    <xsl:sequence select="$x"/>
  </xsl:template>

</xsl:stylesheet>

Fun, isn't it? :-)

Wednesday, April 2, 2008 5:45:28 AM UTC  #    Comments [0] -
xslt
# Monday, March 31, 2008

I was thinking earlier about the difference between named tamplates and functions in xslt 2.0 and have not found satisfactory criterion for a decision of what to use in each case. I was not first one who has troubled with this, see stylesheet functions or named templates.

To feel easy I deliberately have decided to use functions whenever possible, avoid named tamplates completely, and use matching templates to apply logic depending on context (something like virtual function). I've forgot about the issue until yesterday. To realize the difference one should stop thinking of it, quite opposite she must start solving practical xslt tasks, and if there is any difference, except syntactic, it will manifest itself somehow.

To make things obvious to those whose programming roots are in a language like C++ I shall compare xsl:function with free standing (or static) C++ function, and named xsl:template with C++ member function. In C++ you can use both free standing and member functions interchangeably, however if there is only one argument (among others) whose state transition this function represents then it's preferrable to define it as a member function. The most important difference between these two type of functions is that a member function has hidden argument "this", and is able to access its private state.

Please, do not try to think I'm going to compare template context item in xslt 2.0 with "this" in C++, quite opposite I consider context item as a part of a state. I'm arguing however, of private state that can be passed through template call chain with tunnel parameters. Think of a call tunneling some state (like options, flags, values), and that state accessed several levels deep in call hierarchy, whenever one needs to. You cannot do it with xsl:function, you cannot pass all private state through the function call, you just do not know of it.

This way my answer to the tacit question is:

  •  use xsl:function to perform independent unit of logic;
  •  use named xsl:template when a functionality is achieved cooperatively, and when you will possibly need to share the state between different implementation blocks;

After thinking through this, I've noticed that such distinction does not exist in XQuery 1.0. There is no tunneling there. :-)

Monday, March 31, 2008 6:54:22 AM UTC  #    Comments [0] -
xslt
# Tuesday, March 25, 2008

In the xslt world there is no widely used custom to think of stylesheet members as of public and private in contrast to other programming languages like C++/java/c# where access modifiers are essential. The reason is in complexity of stylesheets: the less size of code - the easier to developer to keep all details in memory. Whenever xslt program grows you should modularize it to keep it manageable.

At the point where modules are introduced one starts thinking of public interface of module and its implementation details. This separation is especially important for the template matching as you won't probably want to match private template just because you've forgotten about some template in implementation of some module.

To make public or private member distinction you can introduce two namespaces in your stylesheet, like:

For the private namespace you can use a unique name, e.g. stylesheet name as part of uri.

The following example is based on jxom. This stylesheet builds expression from expression tree. Public part consists only of t:get-expression function, other members are private:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:t="http://www.nesterovsky-bros.com/public"
  xmlns:p="http://www.nesterovsky-bros.com/private/expression.xslt"
  xmlns="http://www.nesterovsky-bros.com/download/jxom.zip"
  xpath-default-namespace="http://www.nesterovsky-bros.com/download/jxom.zip"
  exclude-result-prefixes="xs t p">

  <xsl:output method="text" indent="yes"/>

  <!-- Entry point. -->
  <xsl:template match="/">
    <xsl:variable name="expression" as="element()">
      <lt>
        <sub>
          <mul>
            <var name="b"/>
            <var name="b"/>
          </mul>
          <mul>
            <mul>
              <int>4</int>
              <var name="a"/>
            </mul>
            <var name="c"/>
          </mul>
        </sub>
        <double>0</double>
      </lt>
    </xsl:variable>

    <xsl:value-of select="t:get-expression($expression)" separator=""/>
  </xsl:template>

  <!--
    Gets expression.
      $element - expression element.
      Returns expression tokens.
  -->
  <xsl:function name="t:get-expression" as="item()*">
    <xsl:param name="element" as="element()"/>

    <xsl:apply-templates mode="p:expression" select="$element"/>
  </xsl:function>

  <!--
    Gets binary expression.
      $element - assignment expression.
      $type - expression type.
      Returns expression token sequence.
  -->
  <xsl:function name="p:get-binary-expression" as="item()*">
    <xsl:param name="element" as="element()"/>
    <xsl:param name="type" as="xs:string"/>

    <xsl:sequence select="t:get-expression($element/*[1])"/>
    <xsl:sequence select="' '"/>
    <xsl:sequence select="$type"/>
    <xsl:sequence select="' '"/>
    <xsl:sequence select="t:get-expression($element/*[2])"/>
  </xsl:function>

  <!-- Mode "expression". Empty match. -->
  <xsl:template mode="p:expression" match="@*|node()">
    <xsl:sequence select="error(xs:QName('invalid-expression'), name())"/>
  </xsl:template>

  <!-- Mode "expression". or. -->
  <xsl:template mode="p:expression" match="or">
    <xsl:sequence select="p:get-binary-expression(., '||')"/>
  </xsl:template>

  <!-- Mode "expression". and. -->
  <xsl:template mode="p:expression" match="and">
    <xsl:sequence select="p:get-binary-expression(., '&&')"/>
  </xsl:template>

  <!-- Mode "expression". eq. -->
  <xsl:template mode="p:expression" match="eq">
    <xsl:sequence select="p:get-binary-expression(., '==')"/>
  </xsl:template>

  <!-- Mode "expression". ne. -->
  <xsl:template mode="p:expression" match="ne">
    <xsl:sequence select="p:get-binary-expression(., '!=')"/>
  </xsl:template>

  <!-- Mode "expression". le. -->
  <xsl:template mode="p:expression" match="le">
    <xsl:sequence select="p:get-binary-expression(., '<=')"/>
  </xsl:template>

  <!-- Mode "expression". ge. -->
  <xsl:template mode="p:expression" match="ge">
    <xsl:sequence select="p:get-binary-expression(., '>=')"/>
  </xsl:template>

  <!-- Mode "expression". lt. -->
  <xsl:template mode="p:expression" match="lt">
    <xsl:sequence select="p:get-binary-expression(., '<')"/>
  </xsl:template>

  <!-- Mode "expression". gt. -->
  <xsl:template mode="p:expression" match="gt">
    <xsl:sequence select="p:get-binary-expression(., '>')"/>
  </xsl:template>

  <!-- Mode "expression". add. -->
  <xsl:template mode="p:expression" match="add">
    <xsl:sequence select="p:get-binary-expression(., '+')"/>
  </xsl:template>

  <!-- Mode "expression". sub. -->
  <xsl:template mode="p:expression" match="sub">
    <xsl:sequence select="p:get-binary-expression(., '-')"/>
  </xsl:template>

  <!-- Mode "expression". mul. -->
  <xsl:template mode="p:expression" match="mul">
    <xsl:sequence select="p:get-binary-expression(., '*')"/>
  </xsl:template>

  <!-- Mode "expression". div. -->
  <xsl:template mode="p:expression" match="div">
    <xsl:sequence select="p:get-binary-expression(., '/')"/>
  </xsl:template>

  <!-- Mode "expression". neg. -->
  <xsl:template mode="p:expression" match="neg">
    <xsl:sequence select="'-'"/>
    <xsl:sequence select="t:get-expression(*[1])"/>
  </xsl:template>

  <!-- Mode "expression". not. -->
  <xsl:template mode="p:expression" match="not">
    <xsl:sequence select="'!'"/>
    <xsl:sequence select="t:get-expression(*[1])"/>
  </xsl:template>

  <!-- Mode "expression". parens. -->
  <xsl:template mode="p:expression" match="parens">
    <xsl:sequence select="'('"/>
    <xsl:sequence select="t:get-expression(*[1])"/>
    <xsl:sequence select="')'"/>
  </xsl:template>

  <!-- Mode "expression". var. -->
  <xsl:template mode="p:expression" match="var">
    <xsl:sequence select="@name"/>
  </xsl:template>

  <!-- Mode "expression". int, short, byte, long, float, double. -->
  <xsl:template mode="p:expression"
    match="int | short | byte | long | float | double">
    <xsl:sequence select="."/>
  </xsl:template>

 </xsl:stylesheet>

Tuesday, March 25, 2008 6:23:30 AM UTC  #    Comments [0] -
Tips and tricks | xslt
# Monday, March 3, 2008

I often find myself in a position that whenever I'm thinking of something, I can find the idea to be already implemented somewhere.

A good example is xslt/xquery -> java code.

Well, the world is full with smart guys. :-)

Monday, March 3, 2008 6:08:21 PM UTC  #    Comments [0] -
xslt
# Thursday, February 28, 2008

Wow, I've found an article Code generation in XSLT 2.0. The article is dated by year 2005.

Well, I was inventing a bicycle. This is a good lesson for me.

I'm going to study very carefully about SQL Code Generation, as this is exacly the same task I'm facing now.

Thursday, February 28, 2008 4:35:24 AM UTC  #    Comments [0] -
xslt
# Wednesday, February 27, 2008

I've updated jxom.zip.

There are minor fixes there. The most important addition is a line breaker. The purpose of the line breaker is to split long lines.

Long lines appear if there are verbose comments, or there is a very long expression, which was not categorized as multiline.

It's not perfect, however looks acceptable.

Now I'm facing a next problem: I need to do a similar job I'm doing to java, however with sql. Moreover, I need to support several dialects of sql. I'm not sure if it's possible (worth) to define single sql-xom.xsd, or should I define sql-db2-v9-xom.xsd, sql-sqlserver-2005-xom.xsd, ...

The bad news are that sql grammar is much more complex than one of java. Probably I'll start from some sql subset. In any case I do not consider generation of sql "directly", as jxom fits remarkably into its role.

Wednesday, February 27, 2008 1:30:47 PM UTC  #    Comments [0] -
xslt
Archive
<May 2008>
SunMonTueWedThuFriSat
27282930123
45678910
11121314151617
18192021222324
25262728293031
1234567
Statistics
Total Posts: 310
This Year: 23
This Month: 3
This Week: 0
Comments: 209
Locations of visitors to this page
Disclaimer
The opinions expressed herein are our own personal opinions and do not represent our employer's view in anyway.

© 2014, Nesterovsky bros
All Content © 2014, Nesterovsky bros
DasBlog theme 'Business' created by Christoph De Baene (delarou)