General Interface Forums and Blogs have Moved

If you didn't already know, as part of the GI Open Source contribution to Dojo Foundation, General Interface forums and blogs are now hosted on www.generalinterface.org.

GI developers and contributors will continue the blogs and forums post on the new site.

You can access the new forums at General Interface Forums

The new blogs are at GI contributor blogs

Here's an RSS URL

Look for news on GI 3.8 releases, GI 3.9 features and latest on GI 4.0 on www.generalinterface.org.

This will be my last blog on TIBCommunity, but not the last blog for GI. So we look forward to seeing all developers on the new General Interface Forums

ActiveSpace Browsers

Well, it has been a little while since my last post, and I think it is very much time to post some new information on ActiveSpace by introducing a new concept that is specific to ActiveSpaces and we believe extermely useful: the Space Browsers.

Space Browsers are a very simple to use yet very powerful feature of AS that allows programmers to implement the following architectural paradigms in order to create truly elastic distributed applications and services:

• Iterating over a subset (view) of the entries stored in a Space

• Continuous Queries

• Using a Space as a distributed Queue

• Grid computing paradigms such as Map/Reduce

Space Browsers (and Listeners) complement the Space’s basic put/get/take/lock methods (which require the user to provide a specific value for the key fields, and therefore operate on a single Space entry at a time) with the ability to ‘iterate’ through sets of entries for which the exact key field values are not known in advance.

A Space Browser is an object that can be seen as a sort of continuously updated iterator over entries stored in a Space. I say ‘sort of’ because, like an iterator, it has a ‘next()’ method which returns the next entry to ‘work’ on, but that’s where the similarities stop.

Browsing over a Space view

Let’s look at the first use case: iterating over a ‘view’ of the entries stored in a Space: when a Browser is created over a Space an optional filter string can be passed that will refine the set of entries being considered by the Browser.

In ActiveSpaces filters (which can be applied to both Browsers and Listerners) follow the SQL-92 syntax and implement many (but not yet all) of its clauses, put simply a filter is the part of a query that follows the ‘where’ statement in a ‘select * from Space where …’ query.

For example if a Space ‘customers’ contains Tuples that have an ‘Age’ field, browsing through all of the entries in the Space for people aged less than 30 years old is as simple as passing the string “Age < 30” as a filter when creating the Browser.

Continuous Queries

However unlike a query in a database that works only on a snapshot of the data contained in the database at a specific point in time, Space Browsers are continuously updated according to the changes in the Space being browsed and can be open-ended, effectively providing ‘Continous Query’ capability of the data contained in the Space. To that extend and unlike a more ‘traditional’ iterator object, the Browser objects do not have a ‘hasNext()’ method, but rather have a timeout value. That value is the amount of time the programmer is willing for the final ‘next()’ call to block for while waiting for something new of ‘next’ on.

Because of the low latency orientation of ActiveSpaces, a blocking 'next()' will unblock and return the newly inserted or updated Tuple in real-time as soon as the change happens, and that no matter how many Listeners or Browsers may be interested in that new Tuple.

Using a Space as a distributed Queue

This continuous real-time updating of the Browser also means that changes happening to the data in the space are automatically reflected on the list of entries about to be browsed as they happen: a Space Browser never gives the user outdated information!

For example, if an entry existed at the time the Browser was created, but this entry gets taken from the space before the Browser’s ‘next()’ method gets to it, then this entry will not be returned by the ‘next()’ method.

Also, unlike a traditional iterator that only allows its users to look at a series of data items, Space Browsers allows users to not only look but also operate on entries in an iterative manner. To that extend Browser have a ‘type’ that influence the type of operation applied to the entry when the Browser’s ‘next()’ method is invoked:

• The GET Browser’s ‘next()’ method does a ‘get()’ on the next entry to browse (very much like a regular iterator).

• The TAKE Browser ‘next()’ method however does a ‘take()’ on the next entry (that is, it atomically retrieves AND removes the next entry currently availbe to take from the space).

• The LOCK Browser ‘next()’ method does a ‘lock()’ on the next entry to browse (that is, it atomically retrieves AND locks of the next entry currently available to lock in the Space).

Because of the fact that Browsers are continuously updated in real-time according to the changes in the unerdlying Space, TAKE and LOCK Browsers effectively (and very simply) allow 1-of-n ‘consumption’ of the entries stored in a Space. No matter how many TAKE Browsers may be created over a Space (even if those Browsers are created by many separate processes deployed on many separate physical hosts), what is taken from the Space by one Browser’s ‘next()’ method will NEVER be taken by another Browser’s ‘next()’ method, thereby allowing the programmer to use a Space like one would use a store-and-forward messaging Queue (for temporal and localization de-coupling, analoguous to the ‘master/worker’ pattern in Space-based architecture for example), except that in our case the Tuple ‘Queue’ being a Space, it can be distributed and scalled up as needed.

Writing elastic applications and services made easy

Creating a distributed service is therefore extremely easy to program using ActiveSpaces, clients simply put requests in a ‘request’ space, and servers simply create a TAKE (or LOCK) Browser on the Space, when ever a new request is put in that space by a client, it is automatically taken by one (and only one) of the servers for processing (and the reply Tuple can be put in a ‘reply’ Space), all the implementer of that service has to do is invoke ‘next()’.

The code below is a copy-and-paste of the part of the ActiveSpaces example server-side program that shows this feature, in this example the program waits for a request to be put in the 'request' space (by the client-side example program) and 'processes' it by adding a new field to the request tuple and putting this updated tuple into the 'reply' space. If there is no new requests to process after 500 milliseconds the program just prints out a status of the number of requests it has processed so far and goes back to waiting for more requests.

        // Create a Take Browser on the request space with a timeout of 500 milliseconds
        BrowserDef browserDef = BrowserDef.create().setTimeout(500);
        Browser takeBrowser = null;
        try {
            takeBrowser = metaspace.browse(space_req.getName(), BrowserType.TAKE, browserDef);
        } catch (ASException e) {
            System.out.print("Problem creating the take browser: " + e);
            return;
        }
 
        // the TakeBrowser allows the use of the 'request' space as a queue 
        Tuple tuple = null;
        SpaceEntry entry = null;
 
        do {
            // Try to take a tuple from the space
            try {
                entry = takeBrowser.next();
                // if next() returned an Entry, we successfully took/consumed the entry from the Space
                if (entry != null) {
                    tuple = entry.getTuple();
                    // add the replyer field to the tuple and put it into the reply space
                    tuple.put("replyer", selfmember);
                    space_resp.put(tuple);
                    served++;
                    if (served % 100 == 0) {
                        System.out.println("I have serviced " + served + " requests");
                    }
                } else // there is nothing to take: the next() timed out and returned null
                {
                    System.out.println("I have serviced " + served + " requests and I am waiting for more");
                }
            } catch (Exception e) {
                System.out.println("Exception in the take browser: " + e);
            }
        } while (true);

New server instances can be added (or removed) on the fly, without service interruption, thereby allowing ActiveSpaces users to create ‘write-once, deploy as many as you need’ elastic applications.

Subscription with 'Initial Values'

But that’s not all of what you can do with a Space Browser: Browsers (and Listeners) have two associated scopes that can further refine (than a filter) the set of entries being browsed.

The ‘time scope’ can be used to narrow down the period of time of interest:

• 'SNAPSHOT' means that the browser starts with all the entries in the space at the time the browser is created (or initial values), but is not updated with new entries that are put into the space after that moment.

• 'NEW' means that the browser starts empty, and is updated only with entries (or associated events) put into the space after the moment of the browser’s creation.

• 'ALL' means that the browser starts with all the entries in the space, and is then seamlessly continuously updated with new entries as they are added into the Space. Readers familiar with pub/sub messaging systems will note that creating a Listener with a time scope of ‘ALL’ allows then to receive ‘initial values’ before they start receiving future publications.

Grid computing programming made easy

The other scope of Browsers and Listeners is the ‘distribution scope’ and it can be used to narrow down the set of entries according to their distribution over the Space's seeders:

• ‘ALL’ means that all of the entries stored in the Space will be considered, regardless of where they are being seeded

• ‘SEEDED’ means that only the entries assigned to the local metaspace member on which the Browser/Listener is created will be considered.

What may not be obvious right away is that the distribution scope of ‘SEEDED’ is the gateway to easy creation of grid-computing applications using (for example) a Map/Reduce pattern.

The Map/Reduce pattern can be described as follows: a very large data-set is divided in a set of Key/Value pairs (or in the case of ActiveSpaces in a set of Tuples) that are evenly distributed over a set of nodes (i.e. a Space’s seeders): this is the ‘mapping’ phase. Each node then processes the subset of the data that was assigned to it and puts the result of it’s processing (a smaller or equal set of Tuples) in another Space for the client to retrieve (or for another set of nodes to process): this is the ‘reduce’ phase. Most grid computing applications can be architected as a series of ‘Map’ and ‘Reduce’ phases.

This type of grid computing applications is very easy to create using ActiveSpaces: ‘mappers’ are simply nodes that join a ‘mapping’ Space as seeders each one creating on that space a Browser with a distribution scope of ‘SEEDED’ and just need to invoke ‘next()’ on that Browser: every time a new subset of the data is mapped to a specific node, that node’s ‘next()’ method will return with the new tuple to process. Alternatively a ‘control’ Space could be used to coordinate which mapping or reduce phase of the overall process is currently being executed if a more ‘batch’ rather than ‘event-driven’ type of processing is desired.

Conclusion

In conclusion, while I didn’t even describe completely all of the features of the Space Browsers and Listeners (for example the EventBrowsers), I hope I will have been able to show the power of the Browser concept and the various ways they can be used, and how they greatly simplify the work of the application programmer allowing them to very easily create elastic distributed/grid-enabled applications and services.

Welcome to my OSGi Blog @ TIBCO.

If you are looking for TIBCO specific information about OSGi, then I have to disappoint you. This blog is meant for communicating Enterprise Export Group related information to the greater OSGi community. The EEG is open only to OSGi Alliance members and it is up to the group members and especially the chair-men to open up the communication and information flow.

You may have heard that my co-chair since the inception of the EEG in late 2006, Eric Newcomer, has moved on to new responsibilities, which unfortunately do not allow him to stay involved in the OSGi work. I am very grateful to Eric for his help and mentorship.

However, we are very happy to have found an excellent successor in David Bosschaert from Progress Software.

Ok, back to business.

For the last three years the Enterprise Expert Group has worked on integrating mostly J* specs with OSGi in a manner that makes it easy for JEE developers to run existing JEE applications in OSGi while at the same time with only minimal adjustments to the design to allow for making optional use of the unique OSGi capabilities. One example of this adaptation is JNDI. For the JNDI spec, we have come up with an elegant way of using JNDI capabilities through services, while at the same time allowing for existing hard wired application to run in OSGi in "traditional" mode. You may also read this as "classic" or "legacy" depending on where you stand on the JEE vs. OSGi debate. In a similar fashion we have worked out the integration of JPA. Others map easily and elegantly without a lot of complications, e.g. JTA and JDBC.

In addition, in the enterprise profile we introduce new capabilities to OSGi, which are not driven by JAVA specs, i.e. Remote Services and Remote Service Admin Service. These powerful concepts allow for scaling of OSGi services across JVM boundaries and open up a whole new set of designs and use cases.

What is the Enterprise Profile now?

Glad you asked. Let me start by saying what it is not:

It is not an all-or-nothing specification document that requires compliant implementations to support every service listed. As such there is no single TCK that can be run against an implementation to claim compliance.

Instead, the profile is a collection of services from the core, compendium, as well as new enterprise driven services, which will ultimately be part of the compendium itself. The Enterprise Profile is the first point of reference for enterprise developers looking for guidance in the vast amount of spec documents. The first question someone new to OSGi typically is driven by is

'What is relevant to me? Do you expect me to read the entire 400 pg. core spec plus the 800 pg. compendium stack before I can do anything useful with it?'.

The introduction chapter to the spec is trying to put the services in context with typical use cases that developers can follow and identify themselves with. The following chapters are literal copies of the relevant chapters from the core and compendium specs for easy reference.

When you download the current draft you will notice that the list of included services seems incomplete. Indeed it is. The final version will also include already published and final specifications like Configuration Admin Service and Blueprint.

Where can you find the public Enterprise Profile?

The public interim draft can be found at the public OSGi web site for download by everyone.

In the near future we plan to release the final version and make it available for the general public. We encourage the OSGi community to take advantage of it, try it out, and provide valuable feedback about what works and what does not.

Stay tuned for more updates of the EEG at this location.

Tim.

Here is a quick status update on the spec for TIBCO BusinessEvents 3.0:

- graphical models: concept, event (UML Class), state (UML State)

- graphical reports: project analyser/debugger

- event processing languages: rules language, BusinessEvents Query Language (BQL)

- event processing agent types: inference node, query node, cache node

- architectures: any combination of agents in any processing order

- event persistence: in-memory, cache (Coherance), checkpoint (BerkeleyDB), backing store (Oracle), any combination thereof

- rule engine capabilities: inference (Rete-type), defined statically (project-level code persistence), compiled to bytecode

- query language capabilities: static and continuous queries, defined dynamically (run-time code compilation)

- dashboard: provided through Syndera, RTView, GE / AJAX UIs

- business user interface: Decision Manager (for decision tables) BRMS

- business user interface workflow control: Rule Management System, customisable

- standard channels: EMS, RV, internal

- standard interfaces: DBMS (Oracle DBConcept), BW

- packaging: with and without state model / engine (Enterprise, Inference editions)

A couple of days ago I posted a comment to the general TIBCOmmunity regarding virtualization. However, I feel it was not the appropriate place to publish my observations on virtualization. I am including the text in this blog, because of the ramifications virtualization has on architectural decisions.

Virtualization: Hyper-V, VMware, and Xen

Over the past year I have been involved in several project that use VMware, and I have used Hyper-V on my own systems. I do not have any experience using Xen, so I will not go into detail about it.

Virtualization In General

While VMware, Xen, and Hyper-V offer the benefit of better hardware utilization and easier management, there is also a downside to using virtualization. Performance is the major drawback. Processing seed is reduced between 10% to 25% (VMware claims 15% reduction) , and I/O (mainly network) can be reduced by as much as 50%. All of the vendors have tuning guides, and you should follow the suggestions in these guides if you want to achieve the best performance possible.

All virtualization hosts require resources to manage the guest operating systems. If all of the CPUs are used by guest OSs and the host needs to do some processing it will swap out one of the guest OSs and take command of one of the CPUs. Once the host is finished processing the gust will return to normal. This delay is typically less than a second, but if you have latency sensitive operations in the guest OS you should make sure that there is at least one free CPU for use by the host OS. Also, I would reserve around 1GB of RAM for the host OS.

Programs that use a lot of disk I/O like large databases, and programs that require very low latency should not be put on virtualized guest OSs. I also would advise not putting EMS or RVRD on a virtual machine. With EMS I have seen as much as 75% decrease in performance running on VMware ESX 3.5 Update 3.

Microsoft Hyper-V

Hyper-V supports a very broad set of hardware. Just about any system with 4GB or RAM and a CPU that supports Intel VT or AMD-V will work. The performance of Window Server 2003 R2, and Windows Server 2008 guest OSs are excellent. I have found that once everything was tuned the CPU reduction was between 10% and 15%, and the network performance was only reduced 10%.

In Windows Server 2008 the management applications are not very good, but that should improve in Windows Server 2008 R2. However, it still has a long way to go before it will have the feature list of VMware.

Although Microsoft claims to support Windows Server 2000, Windows XP, and SUSE Linux Enterprise Server 10 SP1/SP2, the support for these OSs are pretty poor, and I would suggest not running them on Hyper-V.

VMware ESX

VMware ESX is by far the most popular virtualization host. It also offers the most management features. VMware ESX requires an exact set of hardware to work. The complete list is on the VMware’s web site, but must hardware vendors will offer servers that are specifically designed to work with VMware ESX. ESX 3.5 and ESX 4.0 offer good guest CPU and network performance. I have seen and reduction of 15% to 20% in CPU performance and a reduction of 15% to 20% in network performance.

Version before ESX 3.5 Update 2 did not have good network performance. With ESX 3.0.2 I experienced an I/O reduction of more than 60%. I highly advise running TIBCO applications only on ESX version 3.5 Update 2 and above. The new ESX 4.0 promises many performance improvements, but I have not run any test on it yet. We will have more information in November after TIBCO India completes their performance tests on ESX 4.0.

VMware ESX supports a large number of guest OSs, With Windows Server and Linux being the most common. However, Red Hat Enterprise Linux version earlier than 4.7 and 5.2 have a interrupt timing issue with ESX, and should be avoided. Even with RHEL 4.7, 5.2, and greater you have to put divider=10 as a boot parameter in GRUB to alleviate this timing issue. Please follow the instructions in the VMware guide “Installing and Configuring Linux Guest Operating Systems”. You can find it on the VMware support web site. Although VMware publicly says this is just an issue with the machine time, it actually causes problems with I/O interrupts drastically reducing performance. Therefore, do not run RHEL guests unless you have the divider=10 in the GRUB.

The new VMware vShpere (ESX 4.0) suite offers many new exciting features like active fault-tolerance and site recovery manager to name a few. However, these features have numerous limitations and you need to look at the fine print. For example, the new active fault-tolerance feature will only work on guest OSs that use one CPU, and the site recovery manager requires special hardware.

Conclusion

Virtualization is great of management and hardware utilization, but it has its limitations. Make sure you know these limitations when architecting a solution, and beware of the performance and latency implications.

I decided to create this blog because it's an easy way to share my thoughts with the TIBCO community. I know the title is a bit strange, "Service Oriented Architecture Architecture", but I wanted to emphasize this blog is about architecture and not technical details of SOA implementation or product information. Of course you have to know the capabilities of the various products to create a successful architecture, so I will mention products in this blog, but I will only focus on the way they affect architectural decisions.

Regards,

A. Kevin Bailey

Enterprise Architect

CTS-EMEA-ASG

The simplest way I can explain ActiveSpaces is with this single sentence:

• ActiveSpaces is a distributed middleware that offers the functionality of a database and of a messaging system in a single interface.

In essence, ActiveSpaces is what you get when you Mashup a database and a messaging system together. Of course, a 'one-liners' like this may sound like a bold claim, and I certainly won't claim that it implements every single feature offered by any of the existing database and messaging systems, but read on and I hope to be able to explain that it implements the features most applications need most of the time.

Databases are good at storing or looking up data, and using stored procedures you can even validate or eventually transform the data in an event-driven manner as updates happen, but you are ultimately limited by the fact that the database server is a non-distributed shared resource that is expensive to scale. And because client applications can not be proactively notified of changes in the data they are interested in by the server it is very difficult or expensive to create elastic distributed event driven applications using only a database (even in association with a cache).

Messaging systems are very good at distributing data and events through publish/subscribe and therefore greatly simplify the creation of scalable distributed applications, but publish/subscribe can induce unwanted temporal coupling because the receiving application has to be on-line and subscribing at the time a message is published in order to receive it. Message queues and guaranteed messaging can be used to avoid this problem and provide the temporal independence that you naturally get from a database by 'buffering' data, but only temporarily: messages are forgotten as soon as they are consumed. Ultimately, a messaging system is a store-and-forward 'data pump' designed to distribute data, not hold it as a system of record for later perusal.

So what happens is that in order to achieve a scalable architecture those that can afford it (or have no other choice) use both a database and a messaging system at the same time. But even then, besides the potential inefficiencies and operational complexity, there is a lot of development work to be done to deal with the impedance match between the two worlds: for example coordinating data models or translating between the data formats, or dealing with the 'iterator driven' versus 'callback driven' orientation of each interface (not to mention the synchronization of the transition between the results of a query and the reception of update messages).

ActiveSpaces offers all of those features and none of the headaches because everything is integrated in a single easy to use interface:

• Similarly to a database you can put, get and even take (atomic get and remove) rows. You can either get a single row back by getting a tuple by it's key field values, or use a SpaceBrowser to view a filtered subset of a space.

• Like with a messaging system, you can be notified of changes in the data proactively and immediately using Listeners.

But the Mashup does not stop here...

• Tuples are self-describing and platform independent map of fields, just like messages.

• Unlike database queries, SpaceBrowsers are not just static 'snapshots' of the data, they are continuously updated according to the changes in the Space, as those changes happen.

• Unlike when subscribing with most messaging systems you can request to get 'initial values' and specifying a filter query when creating a Listener.

• SpaceBrowsers can also be used to monitor events through an iterative rather than callback driven interface

• Concurrent modification of the data can be controlled using either 'Compare and Set' operations or locking or even transactions (optimistic or pessimistic control).

• And of course, let's not forget that ActiveSpaces itself is an embeddable peer-to-peer distributed system...

There's of course a few other interesting 'details' that I will expand on in future posts, and remember that I am only describing version 1.0 of ActiveSpaces ;). But the important message that I hope to have conveyed here is that picking the most useful features of database and messaging systems in order to create a single 'best of both worlds' interface is the underlying theme of ActiveSpace's feature set, and the key to distributing (and therefore elastically scaling) enterprise applications.

Hello and welcome to the ActiveSpaces blog!

ActiveSpaces is TIBCO's upcoming Data and Messaging Grid product. Technically ActiveSpaces is a distributed in-memory implementation of the Tuplespace concepts.

Hiding beyond this mouthful of technical terms, ActiveSpaces is a fundamental part of the infrastructure that will enable what is described by TIBCO's CEO as the Enterprise 3.0.

A Tuplespace (typically simply referred to as a "Space") can be seen as a form of distributed shared memory: an entity in which data (expressed as Tuples (i.e. 'database rows')) can be concurrently access by multiple processes. Because the data is stored in memory, storage and retrieval of the Tuples is faster than when using a disk-based system. And because it is distributed, it scales according to the number of participants.

ActiveSpaces is a middleware that greatly simplifies the job of Enterprise application developers, providing them with an easy to use interface for data storage and retrieval (Data Grid) and process coordination (Messaging Grid) and allowing them to concentrate on implementing the business logic. Using "Space Based Architecture" (SBA), developers can very easily create distributed, event-driven and scalable (i.e. "Elastic") applications.

On GI Performance Tuning

Performance is a big topic. A while ago I saw a blog on GI performance turning, see Sunil's blog, , which hits on some basic point of AJAX application optimization. The GI documentation also devotes two entire Chapters on this topic (Optimizing Application Performance and Optimizing Load Performance ). There's also a video tutorial on this Video 6: Optimize Performance. So, as you can see this is an important topic and a frequently asked question.

Basically there are 3 major type of performance issues that concerns GI application

1. 1 Resource loading performance

2. 2 HTML rendering time performance

3. 3 Data parsing and handling

For the first type of issue, the answer in a nut shell is "asynchronous, asynchronous, asynchronous". However this very AJAX answer is not the whole picture.

Before GI 3.6, XML resources on Internet Explorer are loaded by GI using native MSXML3 Document ActiveX object's load method, which is not same as XMLHTTPRequest object. From version GI 3.3 to 3.5, GI forces the use of cache XML when a cache copy is present and regardless of the HTTP cache control metadata( Understanding IE Caching and GI 3.3-3.5. ) This was done as a workaround for an IE6 defect. From GI 3.6 and later, GI uses the XMLHTTPRequest to load XML resource and thus subject to the same caching rules.

What is not known to some developer is that resouces loaded over HTTP are not cached by the browser if it is not properly configured, expiration time and HTTP cache control are key to the use of browser cache.

What is often forgotten is that regarless of whether your application is AJAX enabled or not, it is in the end still a web-based application. This means that the standard HTML page optimization techniques still applies. These are things like (as found by Yahoo performance team, these are my top 4):

1. 1 Add Expires header (and or HTTP Cache control header)

2. 2 Gzip components

3. 3 Minify Javascript

4. 4 Configure ETags

See the full list at Yahoo performance rules, and the Special Rule for AJAX Make AJAX Cacheable.

As for the last type of issue on Data handling, what is often forgotten by AJAX developers is that the browser (especially older browser like IE6) was never designed to run a full fledge application or handle hundreds of megs of data. So developer should remember to keep the heavy lifting on the server and allow the the client GI application to fetch only the neccessary data when it's needed.

Using hide / show, (setting "visibility")

One of the optimization technique I've seen for HTML rendering is to use the "Visibility" property. What the developer should know about this technique is that GI has a dual DOM architecture. This means that GI is architected to use a secondary abstraction DOM used to model and keep track of GI objects. So making the object "hidden" does not remove it from the GI DOM model. On top of that, while the HTML is not visibly rendered, they are still present in the HTML DOM, which means that any DOM operation on the element will still be affected. In short, this technique should not be applied blindly everywhere. A good example is the the Tab UI control which employs this technique to hide/show the tab panes. A bad example is where the application load every canvas in an application and hides them, regardless of whether the compnents/canvas will ever be shown.

TBD : how application layout and flow design can affect performance, good and bad.

Here's a quick announcement that might be of interest to those of you reading our blog.

We have recently been informed that our most favorite CEP product, TIBCO BusinessEvents, has been nominated, and by virtue of independent judging and votes from the public, has now made it to the Finals of the American Business Awards aka The Stevies.

So if, like me, you have a strong affinity for BE, or would like to see any CEP product gain more positive non-industry exposure, you ought to head over to cast your vote at:

The category is for - New Product or Service of the Year - Computer Software, New Version: TIBCO CEP BusinessEvents 3.0

Final judging begins today through June 1st. I'm told that the race in these categories is tight and every vote counts.

The ABA plans to publicly announce the winners on Tuesday, June 9. The winners will be invited to attend the 7th annual American Business Awards banquet in New York on Monday, June 22 to accept their special People's Choice trophy(s).

We will keep you posted on any developments.

In simple terms, Business Process Management is the capacity to completely manage the end to end lifecycle of the business process. The intent of BPM is to identify core business processes in an organization, automate them with efficient integration to the underlying systems and people and then maximize the process for efficiency. When the most important business processes in an organization are executing at peak capacity, the organization is bound to succeed. BPM has the power to spell the difference between success and doom for an organization.

This popular diagram below efficiently summarizes what components exist in end to end BPM:

It’s worthwhile considering the user persona while looking at each of the blocks in this diagram, just to understand the user needs when you consider a BPM offering.

Modeling and Simulation (includes Rules Design),

Persona: Business Analyst (TIBCO Business Studio):

Previous to BPM, a business process could very well lie as an undocumented artifact in minds of a few experts within the organization. It may trickle down through power point presentations, spreadsheets and word documents. However, the need to manage this business process results in a need to extract the process from all these resources and model it visually and document it in a single location.

The user who designs the business process is typically a business analyst – a persona who doesn’t want to get bogged down by technical details and just wants to model the process at a very high level. The user probably wants to simulate the process as well to identify bottlenecks and figure resource allocation before execution. Rules need to be designed as well in addition to the process and it is imperative that the set up be such that the rules are decoupled from the business process, as the two need to change independent of each other.

There are other requirements such as reporting for easy back and forth between the business user and the rest of the players, revision control as the process. Also the capacity to import business processes from third party modeling environments. Lastly, this business process needs to be handed off easily to a process developer for implementation with ease and no loss of information during the transfer.

Implementation

Persona: Process Developer (TIBCO Business Studio):

The Process Developer wants to pick up the business process designed in consortium with the business user and implement details. A process developer is a technical person who wants to get the business process ready for execution. For example, he may want to hook up a service step to a webservice call out or to a database, build out forms for user interaction steps, etc. A good point to note is the importance of decoupling the SOA layer from the BPM layer at this point so the two can change independent of each other.

Execution

Persona: Op/Admin (TIBCO iProcess Engine):

Once the process is ready for execution, the op/admin takes over in terms of ongoing operational management of the process. Levels of scale in terms of number of users being logged on simultaneously, number of transactions, capacity to set up the engine for high availability, scalability are all typical requirements for consideration. Other important considerations include the capacity to get visibility at the system process level to get an idea of health of the execution, capacity to pause and resume, upgrade the business process to newer version numbers without interrupting execution. Lastly, the ease with which you can design–implement–test, as you’ll need to execute that implementation lifecycle many times every time before you go into production.

Business Activity Monitoring

Persona: Business Manager (TIBCO iProcess Insight):

The Business Manager is the person who is accountable for the business process. He needs to get his hands on metrics that prove to him how his process is performing on a day to day basis and helps him identify the bottlenecks in the process. Ease and accuracy of reporting, report design and customization, drill down capability to various resources in the process are all critical requirements for this stage.

Optimization

Persona: Business Analyst (TIBCO iProcess Analytics):

One of the primary purposes of BPM is the need to improve the efficiency of the business process by identifying bottlenecks. For this purpose, the business analyst needs to be able to observe long term trends and drill down into various aspects of the business process to identify bottlenecks. In addition, the suite needs to have the capacity to “round trip” or feed back information from the trend analysis back into the process model for redesign to maximize for efficiency.

This white paper leads you into a more in-depth discussion on the various aspects of BPM: Guide through the BPM Maze

Object-Oriented Programming in JavaScript: Packages in TIBCO General Interface

The article, “ Object-Oriented Programming in JavaScript with TIBCO General Interface,” is an overview of the primary object-oriented constructs provided in TIBCO® General Interface™. In this blog post and several others to follow, a more detailed examination of other object-oriented constructs that TIBCO offers. The first object-oriented concept addressed is one sorely lacking in the JavaScript language but provided in TIBCO General Interface: packages.

A package, or namespace, is the construct that object-oriented languages use to organize related classes into a common group. Packages promote good code organization and prevent accidental name collisions by providing uniqueness to class names.

TIBCO General Interface goes one step further than other object-oriented frameworks in providing the ability to declare the package with code-defining methods and attributes relating to it.

In TIBCO General Interface, a package can be identified using the dot-separated reverse domain name convention; Unlike Java, it does not however need to be associated with a directory in the file system with the same name. GI framework does make use of package and defines them in a file named Package.js (like jsx3/lang/Package.js and jsx3/util/package.js), however this is an abitrary code organization choice and not a requirement.

The WSDL Mapping 2 sample shows code in a package named eg.WSDL2, which holds all static methods for the sample.

The Package class provides an introspectable API for JavaScript/JSX packages. It also provides a way of defining new packages.

An instance of this class may be obtained in one of the following ways (this class may not be instantiated directly):

• >> jsx3.lang.jsxpackage

• >> jsx3.Package.forName('jsx3.lang')

In this example, the JavaScript object jsx3.lang is known as the package "namespace," which is a plain JavaScript object and is analogous to the constructor property of a jsx3.Class.

The following is an example of how to define a new package called eg.tests:

jsx3.lang.Package.definePackage(  "eg.tests",   // the full name of the package to create  
function(tests) {                  // name the argument of this function "eg"
    // define a static method like this:
    tests.staticMethod = function() {      ...    };
    // define a static field like this:
    tests.STATIC_FIELD = "...";  }
 
);

In line 1 of the above example, the static method definePackage() on the jsx3.lang.Package class is called with two arguments: the name of the package and an initialization function for defining the package’s members.

Unlike classes, the package and its members can only be static in nature. Although you cannot instantiate a package, you can still reference it using the thiskeyword from within its methods. If any intermediary packages have not yet been defined when calling the definePackage() method, they are automatically defined in the creation of the new package. Furthermore, the definePackage() method can be called many times for the same package. Each call adds to the existing package rather than replacing the package’s existing members. The declaration of the package is optional, except where you desire introspection, which itself is subject for another in depth discussion.

TIBCO General Interface recommends that applications refrain from creating global methods and attributes in JavaScript, which in effect modify the window object. Doing so can cause name collisions and confusion when multiple projects are brought within the same JavaScript server. Rather, if global methods are necessary, add them to packages, instead.

For a fuller example of Package, consider the WSDL Mapping 2 sample.

jsx3.lang.Package.definePackage(
  "eg.wsdl2",                //the full name of the package to create
  function(wsdl2) {          //name the argument of this function
 
   /**
    * Executes the service mapping and subscribes functions to events.
    * call this method to begin the service call (eg.wsdl2.callGetHistoricalQuotes();)
    */
    wsdl2.callGetHistoricalQuotes = function() {
      //executes the service mapping
      var objService = wsdl2.APP.loadResource("sampleGetHistoricalQuotes_xml");
      objService.setOperationName("GetHistoricalQuotes");
 
      //since the server's response (a SOAP document) will be converted to CDF,
      //compile the mapping rules into XSLT for faster conversion
      objService.compile();
      
      //subscribe and call
      objService.subscribe(jsx3.net.Service.ON_SUCCESS, wsdl2.onGetHistoricalQuotesSuccess);
      objService.subscribe(jsx3.net.Service.ON_ERROR, wsdl2.onGetHistoricalQuotesError);
      objService.subscribe(jsx3.net.Service.ON_INVALID, wsdl2.onGetHistoricalQuotesInvalid);
      objService.doCall();
    };
 
   /**
    * {String} stores the feedback from the web service.
    */
    wsdl2.wsOutcome = "Success";
 
   /**
    * {String} stores the message sent from the web service if an error occurs.
    */
    wsdl2.wsMessage = "Error";
 
   /**
    * Repaints the grid control after successful web service call.
    *
    * @param objEvent {String} static field event type for successful response.
    */
    wsdl2.onGetHistoricalQuotesSuccess = function(objEvent) {
       if ( wsdl2.wsOutcome == "Success") {
         wsdl2.APP.getJSXByName("gridResults").repaintData();
         wsdl2.APP.alert("Success","The service call was successful.");
       }
       else {
         // An error was sent from the web service.
         wsdl2.APP.alert("Web Service Error: " + wsdl2.wsOutcome, "The web service sent the following error message: <br/><br/>" + wsdl2.wsMessage, null, "Close", {width:350, height:200});
       }
    };
 
   /**
    * Creates an alert box with HTTP status code on failed service call attempt.
    *
    * @param objEvent {String} static field event type for unsuccessful response.
    */
    wsdl2.onGetHistoricalQuotesError = function(objEvent) {
      var myStatus = objEvent.target.getRequest().getStatus();
      wsdl2.APP.alert("Error","The service call failed. The HTTP Status code is: " + myStatus);
    };
 
   /**
    * Creates an alert box if one or more restrictions fails during message generation.
    *
    * @param objEvent {String} static field event type for message generation.
    */
    wsdl2.onGetHistoricalQuotesInvalid = function(objEvent) {
      wsdl2.APP.alert("Invalid","The following message node just failed validation:\n\n" + objEvent.message);
    };
 
  }
);

Two items of NOTE that is not in the code

1. 1. The sample project uses a package Static member(property) that is not explicitly declared "eg.wsdl2.APP"

2. 2. This package code can be made more portable / reusable by adding a function parameter of objJSX to
   callGetHistoricalQuotes

e.g.

// Add a new parmeter objJSX, which will be the button instance clicked that trigger this action 
wsdl2.callGetHistoricalQuotes = function(objJSX) {
var server = objJSX.getServer(); // app server instane of the GUI object.
// ...
 
objService.setNamespace(server); //Sets the namespace for the server/project to run the service instance within.
 
 
} 

Author: Luke Birdeau

The XML Mapping utility does allow you to specify which records in your CDF document actually get mapped. There is a API call exposed by the jsx3.net.Service class called setCDFRecords that can be called from a mapping rule. For example, assume you have a CDF document in the XML Cache called myXML:

<data jsxid="jsxroot">
<record jsxid="a" checked="1" jsxtext="a"/>
<record jsxid="b" checked="0" jsxtext="b"/>
<record jsxid="c" checked="1" jsxtext="c"/>
<record jsxid="d" checked="1" jsxtext="d"/>
</data>

Next, Create a CDF Document mapping rule using the XML Mapping Utility for the outbound message you are creating. Set its path/value field to: myXML. Add an additional mapping rule (a Script type this time) and set the following as its path/value:

setCDFRecords("record[@checked='1']");

Here is what happened:

Any time a CDF Document mapping is encountered, all record nodes that are immediate children of the root node (data) are placed in a collection (jsx3.util.Collection). Then when a CDF Record mapping is encountered, the first record in the collection is assigned as the CDF Context and the ensuing rules are run. When a CDF Attribute mapping is encountered, the record referenced by the CDF Context is used. The mapper then repeats this as long as there are more records in the collection. By calling setCDFRecords, you effectively tell the system to not use its default query:

currentRecordContext.selectNodes("record");

rather, use a custom query. In your case:

currentRecordContext.selectNodes("record[@checked='1']");

Example We want to map the jsxtext attribute from record with jsxselected checked to WSDL Schema structure

<element name = "PersonSearch">
<complexType>
<sequence>
<element name = "QueryTarget" type = "xsd:string" maxOccurs = "unbounded"/>
</sequence>
</complexType>
</element>

In the Mapping utility, select the rule tree node PersonSearch.

1. 1. Add a CDF Document mapping and enter the name of the CDF Document you wish to map to (its cache id).

2. 2. Next add a Script mapping to the PersonSearch node and set its value to:

setCDFRecords("record[@jsxselected='1']");

Next, select the rule named, QueryTarget.

1. 1. Add a CDF Record Mapping (records don't need a value).

2. 2. Next, add a CDF Attribute mapping and enter jsxtext as the named CDF attribute on that record that you want to map to.

So to recap, you should have two mappings on the PersonSearch rule and two mappings on the QueryTarget rule.

One quite common qu we get is "how" or "when" will BE support the idea of "rule templates", whereby one separates some conceptual prototype rule with runtime instances, and in particular manage these instances separately.

Interestingly, the idea of "production rule" (see http://tibcoblogs.com/cep/2008/11/17/the-value-of-production-rules/ ) somewhat gives you "rule patterns" or "rule templates" for free.

Consider a rule

declare EventA if EventA.cost > 113 then doSomething( EventA).

You can templatize this easily enough to:

declare EventA if EventA.cost > dataPolicy.cost then doSomething( EventA).

But note that dataPolicy could be an object instance, or a scorecard entry, or... another declaration:

declare EventA, dataPolicy if EventA.cost > dataPolicy.cost then doSomething( EventA).

Now you will compare any EventA with any dataPolicy instance. Clearly you will likely have particular dataPolicies for various event types, and can add that as a "join" in the rule condition.

In this way you are effectively using the power of rule declarations to create rule patterns and templates. And of course, as the policies or strategies you are creating for your "master rule" are defined as concepts, then they are also easy to update and manage, for example as "update events" at runtime, or even through running heuristics (expert rules) or S+ analytic functions.

Reassign the "ctrl+s" hot key to "Save & Reload".(Goto Builder Menu "Tools" > "IDE Settings" > "IDE Hot Keys" , locate "Save & Reload" , double click and press CTRL S keys)

Quickly locate a component using "ctrl+click" on a GUI control in the canvas to select the component on the "Component Hiearchy" palette. (NOTE : the matrix component swallows the "ctrl+click". So in order to click select a matrix control, you must do the clik on the little grey patch under the vertical scrollbar where the events bubbling is not cancel)

Float the "Component Libraries" palette to give more display area for the "Properties Editor" and "Event Editor" palettes. (This is really just a preference, but I find it useful since you can see more of the properties at a glance).

Update

Context sensitive help press keys "alt+F1" while focusing on the component you like get help on (For example, component hiearchy tree) to open the HTML Doc page on the component.