Purging Composite Instances : Oracle SOA Suite 11g


To purge all instances in Oracle SOA Suite 11g (11.1.1.4), simply run the following PL/SQL package. Note that the underlying table structure is slightly different in 11.1.1.2 versus 11.1.1.4, so these instructions are specific to 11.1.1.4, but may or may not work on older or later versions.
sqlplus dev_soainfra/welcome1@orcl

DECLARE
  FILTER INSTANCE_FILTER := INSTANCE_FILTER();
  DELETED_INSTANCES NUMBER;
BEGIN
  FILTER.MIN_CREATED_DATE := TO_TIMESTAMP('2001-01-01','YYYY-MM-DD');
  FILTER.MAX_CREATED_DATE := SYSDATE;
  DELETED_INSTANCES := FABRIC.DELETE_ALL(FILTER, 40000000, true);
END;
/

Applicable Versions:

·                     Oracle SOA Suite 11g (11.1.1.4)

Performance Tuning :- Oracle SOA Suite 11g


When installed out-of-the-box, Oracle SOA Suite 11g performs adequately for a development environment. In this post, I describe the key areas you need to focus on to improve your performance.

I have conducted light and heavy load tests on both synchronous and asynchronous Mediator services (not BPEL yet), and my findings and recommendations are documented below.
 

In summary, here are my findings:

·                     Upgrading to PS3 addresses memory instability issues
·                     Size your JVM appropriately
·                     Moving from Sun JDK to JRockit results in a 32% performance improvement
·                     Increasing the Mediator worker threads results in a 30% performance improvement for async services
·                     Changing the audit level from Development to Production results in a 46% performance improvement
·                     Changing the audit level from Production to Off results in a further 61% performance improvement
·                     Tuning the audit configuration causes Production and Off audit levels to perform equally the same
·                     Implementing parallel processing of routing rules may improve performance of Mediator anywhere from 4% to 509%
·                     I recommend using parallel garbage collection, but don't have statistics


1. Apply the PS3 patchset

I cannot stress this enough. Upgrade to SOA Suite PS3 (11.1.1.4). This resolves an enormous amount of memory issues and will save you a lot of pain.

2. Determine the size needed for your Java Heap Space

If you are running AIA Foundation Pack, then realistically you will need to set your Java Heap Space from 6GB to 10GB, depending on the amount of EBOs you are loading and the amount of composites you have. If you are running just SOA Suite, then you could manage with 2GB to 4GB.

1. Start up your SOA server, and wait until all composites are loaded2. Log in to the WebLogic Admin Console (at http://soaserver:7001/console)
3. Navigate to
 Home --> Servers --> soa_server1 --> Monitoring --> Performance
4. Click on "
Garbage Collect"
5. Inspect the
 Heap Free Current
6. Though every environment is different, increase your heap space until you at least have 2 GB available

The Java Heap Space is configured in
$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:
PORT_MEM_ARGS="-Xms6144m -Xmx6144m ...

3. Set your PermSize (for Sun JDK only)

I recommend a min and max of either 1GB or 1.5GB for your PermSize.

The PermSize is configured in
 $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:
PORT_MEM_ARGS="... -XX:PermSize=1024m -XX:MaxPermSize=1024m ...

4. Set your Nursery Size (for JRockit only)

Generally speaking, the Nursery Size should be around 30% of your Java Heap Space.

The Nursery Size is configured in
 $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.shas follows:
PORT_MEM_ARGS="... -Xns2048m ...

5. Use parallel garbage collection (for Sun JDK only)

Garbage collection (and some other recommended settings) is configured in
$MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:
PORT_MEM_ARGS="... -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassUnloading ...

6. Use JRockit instead of Sun JDK

My performance testing has shown an average of
 32% performance improvement when using JRockit versus Sun JDK.

7. Modify and tune the Audit Level

Adjusting the Audit Level and tuning it is necessary.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to
 Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Common Properties --> More SOA Infra Advanced Configuration Properties...
3. Click on "
Audit Config"
4. Set the following values and click on "
Apply" afterwards
AuditConfig/compositeInstanceStateEnabled = false    <-- you must set this for it to work!

AuditConfig/level =
 Production
AuditConfig/policies/Element_0/isActive =
 false
AuditConfig/policies/Element_0/name =
 Immediate
AuditConfig/policies/Element_1/isActive =
 true
AuditConfig/policies/Element_1/name =
 Deferred
AuditConfig/policies/Element_1/properties/Element_0/name =
 maxThreads
AuditConfig/policies/Element_1/properties/Element_0/value =
 10
AuditConfig/policies/Element_1/properties/Element_1/name =
 flushDelay
AuditConfig/policies/Element_1/properties/Element_1/value =
 5000
AuditConfig/policies/Element_1/properties/Element_2/name =
 batchSize
AuditConfig/policies/Element_1/properties/Element_3/value =
 100
My performance tests have shown that changing from Development to Production results in an average of 46% performance improvement (without tuning the audit settings).

My performance tests have shown that changing from
 Production to Off results in an average of  61% performance improvement (without tuning the audit settings). 

My performance tests have shown that if you apply the audit tuning described above, both
 Production and Off perform the same, effectively translating to a 61% performance improvement of the Production audit level.

8. Modify Mediator Service Engine properties (for Mediator only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous Mediator services.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to
 Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Mediator Properties
3. Edit the settings as follows:
Metrics Level:                                        Disabled
Parallel Worker Threads:                        12              <-- experiment increasing this value
Parallel Maximum Rows Retrieved:         600            <-- set to 50x the above setting
Parallel Locker Sleep Thread:                 1                <-- reduces waits for parallel routing rules
My performance tests have shown that changing from 4 worker threads and  200 max rows to 12 worker threads and 600 rows results in an average of 30% performance improvement for asynchronous Mediator services and 0% performance improvement for synchronous Mediator services.

9. Reduce soa-infra Log Levels

Unless required, reduce the SOA Suite Log Levels to error.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to
 Farm_soa_domain --> SOA --> (right-click on) soa-infra --> Logs --> Log Configuration
3. Set all Log Levels to
 ERROR:1 (SEVERE)
4. This must be repeated for all soa_server managed servers in your cluster
 

My performance tests have shown that reducing this from
 NOTIFICATION:1 (INFO) to ERROR:1 (SEVERE) results in an average of  7% performance improvement.

10. Perform Database Tuning

These are the database parameters I used in a large scale implementation. However, remember that they are based on the database AWR reports, so you may not want to just go ahead and apply them blindly. They are here for your reference and have resulted in some improvement in our environment.
AUDIT_TRAIL            = NONE
DB_FILES               = 1024
DISK_ASYNCH_IO         = TRUE 
FILESYSTEMIO_OPTIONS   = SETALL
JOB_QUEUE_PROCESSES    = 10
LOG_BUFFER             = 209715200 
NLS_SORT               = BINARY
SESSION_CACHED_CURSORS = 500
PROCESSES              = 1500
SESSION_MAX_OPEN_FILES = 50
UNDO_MANAGEMENT        = AUTO
PLSQL_CODE_TYPE        = NATIVE
MEMORY_TARGET          = 0
SGA_TARGET             = 6g 
PGA_AGGREGATE_TARGET   = 2g 
TRACE_ENABLED          = FALSE

11. Modify BPEL Process Manager Properties (for BPEL only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous BPEL processes.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to
 Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> BPEL Properties
3. Edit the settings as follows:
Dispatcher System Threads:       10            <-- increase at least from the default value of 2 to 10
Dispatcher Invoke Threads:         20            <-- depends if your targets can handle the load
Dispatcher Engine Threads:        30            <-- should not exceed the sum of the two values above
Payload Validation:                    Disabled
Honestly, these numbers will vary based on the types of process designs you have.


Applicable Versions:

·                     Oracle SOA Suite 11g (11.1.1.4)

Oracle SOA - BPEL Master & Detail Co-ordination Using Signals



Introduction:

A BPEL process can communicate with another BPEL process just like it can communicate with any Web Service – as BPEL processes expose WebService interfaces to the world – or at least to their fellow components in the same Composite Application. When one process – the master in this discussion – calls another one – it can have several types of interaction and dependency on that other process – we will call it the detail process for the purpose of this article: 
  • it is not interested at all in the detail process – its call was a one-way fire and forget
  • it is interested in the response and it will wait for the response before it can continue processing (synchronous calls will always do this, asynchronous calls could have some activity going on while the detail process is churning away)
  • it is interested in the fact that the detail process has reached a certain stage – but it does not actually need a response (it wants a signal but no data)
The Signal and ReceiveSignal activities are Oracle extensions to BPEL – that only work on the Oracle BPEL engine – that help us to implement the third scenario.

As part of the Invoke activity from a BPEL process to another process, we can specify that the called process should be considered a Detail process (and therefore the calling process as the Master process). When we have established this Master-Detail relationship, we can next create a Signal-ReceiveSignal connection between the two. These connections can be created in both directions: the master sends a signal to the Detail (and the Detail waits to receive the signal) and vice versa the Detail process sends a signal that the master is waiting for. Unfortunately, as we will see in this article, we cannot have multiple such interactions between a master-and-Detail pair.

Typical use cases for the signal pattern are situations where a master process can only proceed when detail processes have completed or at least reached a certain state (the master process should only send the email to the customer when the detail process has handed the order to the shipping department) or when a master process calls a detail process to start processing and then needs to do some additional work before the detail process(es) can continue to their next step (master process asks detail to start gathering quotes from car rental companies, than continues to establish the credit-worthiness of the customer and when that has been taken care of indicates to the detail process that it may continue processing).

Note: there is nothing signal .and receiveSignal can do that we cannot also achieve using asynchronous, correlation driven calls. However, when we can achieve our goals using signaling, it is usually much easier to implement and lighter-weight to execute than the full blown correlation based solution.

Implementing BPEL Master-and-Detail processes:

The requirement is our case is that the Master process invokes the detail process to have it start its work. Then the Master performs some additional work and when that is done it signals the detail process to indicate that it may perform the next step of its work. Then the master has to wait for the completion of that work in the detail process. The detail process informs the master of the fact that it is done and the master can continue processing.

We can implement this interaction using Signal and ReceiveSignal.





Step 1:- The Invoke activity that has the check box Invoke as Detail checked. This results in a correlation being created in the BPEL engine between the Master process instance that makes the call and the Detail process instance that is being called.
               partnerLink="DetailProcess.detailprocess_client"
               portType="ns1:DetailProcess" operation="process"
               bpelx:invokeAsDetail="true"/>

Step 2:- The Signal from the Master to the associated Detail instance – allowing the Detail process to continue processing.
               for="'PT30S'"/>

Step 3:- The Receive Signal [from Detail process] in the Master: the master will at that point sit and wait until the signal is received from the correlated Detail process instance.

In this example is the DoImportantMasterStuffThatShouldBeCompleteBeforeDetailProceedsactivity a wait activity that lasts for 30 seconds. The Detail process is implemented as shown in this figure: 



In this case, the 1 indicates the initial creation of the Detail process instance – associated with a specific master instance. The detail process is a one-way service – no response is sent to the master process that invokes it.

The detail process starts with a first activity, ExecuteFirstStagesOfDetailProcessThatIsIndependentOfMaster.

Then at step 2: it reaches a stage where it has to have a signal from its master that is may continue processing. The ReceiveSignal activity is configured to listen for a signal from the master process. While it is waiting, processing stops in the detail process.
                     label="primary"
                     from="master"/>
When the signal has been received, the detail process continues with DoYourDetailThings – a wait activity in this demo that lasts for 10 seconds. When the DoYourDetailThings step it is complete, the detail process sends out a signal to its master.

step 3: Signal_to_master_that_detail_is_done - and continues processing. When we run the Master process, we also get an instance of the detail process. The message flow trace in the SOA console looks something like this:





What is not supported?

I would like to be able to have the master send multiple signals to the same detail process and vice versa. For example to have the master clear the detail process for phase 1, phase 2 and phase 3 of what it does for the master. And something similar for the master – to have t waits for the detail process to reach certain stages in its execution. Here somewhat sophisticated handshake between master and detail. Like the one shown in the next picture: Unfortunately, this does not work.

We cannot have a second signal sent to a detail process – only one Signal (and ReceiveSignal) can be associated with each Invoke of a detail process.
I tried to implement the requirement that the Master process invokes the detail, then performs some additional work and then signals the detail that it may perform stage one. Then the master has to wait for the completion of stage one in the detail process. (So far it all works fine).
These next two additional signal exchanges are not allowed: Then the master does some additional work and subsequently clears the detail process for the execution of its stage 3. The master next waits for a signal from the detail that it has completed stage 2.

Resources:-
Download JDeveloper 11g Application: MasterDetail.zip.
SOA Suite Developer's – Chapter 15