Composite Sensor (sensor.xml) for Oracle SOA

There's a logging feature (Composite Audit Level, under 'Settings') in Oracle's Enterprise Manager Console. However, usually you will not enable it in production environment because the feature captures too much detail(even the payload!) for every transaction and hence could lead to low storage level issue if your "composite" is expected to process too many transactions every minute.

Nevertheless, we still need to capture at least one of two detail for each transaction and such detail can be very useful when coming to problem troubleshooting.

So what do we do without enabling the 'logging' feature? We use the "Composite Sensors". At code level, this composite sensor is a xml with the name "sensor.xml". At Enterprise Manager's level, an composite sensor icon will appear in each transaction line. Just click on that and it will open up a new window that shows the detail.

Run Software in Hidden Mode

Many years ago I was building a Car Workshop Management System. The client is a Car Workshop in Johor. Basically the client uses this software to take order, to track work progress and to generate receipt to customers. The software is able to generate "Account Summary" as well showing the total sales and receipts over a period of time.

One day my boss came to me with a new requirement. The client wanted to have a "secret" mode of the software. Under this secret mode, the client can show to Tax Officers different series of sequence number for invoices and receipts.

So in the software we keep two IDs with running number as the value, one is the "real" invoice number and another one is the "fake" invoice number. So when the client runs the software under "secret" mode, the Account Summary report will show only those transactions with fake number.

Ha ha. What a fun memory!

Oracle SOA Composite SCAC-50012 error during Build

Earlier on my head was spinning for trying to resolve this SCAC-50012 error. JDeveloper throws out this error while building the SOA Composite. 

The compiler suggests me to go to a specific location for the scac.log file. It said the log should contain the detail for the error. 

Well yes there is error detail in the log. It says "could not initialize variable" and "the schema processor cannot find the element " blar blar. So I go ahead and check the blar blar WSDLs + XSDs. They all look perfect. So what the fuck is going on?

So I started all the "long troubleshooting process" which basically means multiple times of "Trial and Error". I removed this WSDL reference, removed that "Invoke Activity". For each change I will compile the whole thing. So I repeated this for every change. Just to narrow down where exactly has gone wrong. 

I did not perform all this in one day as I have other higher priority tasks to take care of. But on and off I will come back to this one as this problem slowly becomes a needle that keeps poking me every time the user asking for an enhancement for this particular SOA Composite. 

So today finally I managed to 'gotcha' the culprit that caused this bloody issue. It is due to the reference to a specific XSD in one of the WSDLs consumed by my SOA Composite is invalid. 

Lesson learnt. Next time the same issue comes to you, please don't "TRUST" the scac.log file. Just examine every WSDLs or XSDs in your Project. Some references may be invalid already.

Dev and Ops

My company's CIO recently came out with an interesting idea.

Before that, let me briefly explain the background for this.

In an organization where the core business is non-IT, the IT department usually gets vendors to help implement an IT solution or product and the vendors should do a KT to the IT department staff so that the IT department staff is able to help maintain the solution or product.

Maintenance usually means :

1) To make sure the solution or product is working well 24x7. If things go wrong, usually it is due to environment issues. For e.g. an version upgrade of one of the products that is also part of the whole solution. Sometimes it could be due to mistakes as the current solution does not read data properly under certain scenarios.

2) To enhance the product or solution to provide more values to the users. Most of the time the values means to save cost for performing one transaction. What kind of cost to save? Maybe to reduce the number of steps for the users to perform a transaction; or maybe to remove some of the manual processes such as "verify data in excel by eyes" thus avoid human error. 

Usually enhancement comes in the form of Change Request. Change Request is very common term used in the IT department of all organizations.

If it is something bigger than a "product/solution enhancement", usually such requirement will be turned into a Project. So the management or PMO will set up a project team for this.

The members of the project shall consist of Project Managers, Solution Architects, Business Analysts, and Developers, and sometimes Testers too.

Every project must be reviewed by Enterprise Architects. According to TOGAF framework, Enterprise Architects should govern the IT architecture of one organization. The value of Enterprise Architect is usually to create "Business Services" to perform certain "Business Processes", and after that the EAs have to define Information Services for each Business Service.

Anyway, my company does not have a EA team right now. As far as I know the management are working on it. When will these come into reality, I don't know.

Let's get back to Project. The problem now is there are so many issues happened to a recent project despite all the proper planning. So CIO and his team decided that:

1) To create a DEV division and a OPS division in one IT team. Take the Middleware team as the example, Middleware has one lead at the moment. In the future there will be a lead for DEV and a lead for OPS. Some developers will be parked under DEV, some developers will be parked under OPS. DEV will need to be held accountable for Project, and OPS will be held accountable for OPS.

This arrangement is not a fixed thing for developers so one developer can jump freely from DEV or OPS and vice versa at anytime.

This arrangement only matters most to the top management so that it is easier for them to check for status with the right person, in this case the lead. If some issues happened in project, look for DEV lead. If something screwed up in operation, look for OPS lead. This is how it works. Top management don't really care if the developer is in DEV or OPS.

Oracle MDS File Does not Exist

Sometimes, after you import a project to your JDeveloper, if that project refers to files in MDS, and complains that the reference files are not available, but if you check in the MDS, the files are actually there.

So usually this is due the MDS that the project refers to does not have the required files. Therefore

1) you either need to put the files in the default MDS referenced by the project, OR
2) you need to make the project to refer to the correct MDS that has all the necessary files. 

For second option, this is what you need to do:

1) Edit adf-config.xml. Make sure yours looks like something like below

2) Restart your JDeveloper.

3) If things did not change, close the Application, then restart your JDev, then reimport that project.

淺談大數據

阅报得知，前公司欲发展“大数据” (Big Data) 工程，以迎合政府的“国家大数据框架”政策。

近几年来，“大数据”是信息工程界的主流。中国风云人物马云最近受访时也说，未来是“数据的时代”，笔者肯定马先生有此一说与大数据技术息息相关。

大数据之所以流行，是因为实时分析海量数据的技术已成熟，重点在于“实时”与“海量”。

换做是从前，要实时为所流入的数据作分析非常困难。往常的作法是，技术人员须写程序，天天按时把数据从不同的“数据库”(Database)集中于“数据 仓库”（Data Warehouse），才能做出所谓的商业分析报告，通常是每日报告，每星期报告，每月报告，每年报告等等，业界称之为商业智能（Business Intelligence），惟就是无法作出“实时报告”。

“大数据”另一特点是可分析来自不同管道的实时数据。比较从前的各种技术，数据须先进入“数据库”后方可作分析。而大数据技术由于可接上不同的“技术协 议”，不必等待数据进入数据库后才能做事。“网民”在网上的一切活动，如使用脸书、推特及其他社交程序，在大数据技术支援下，再配合传统数据搜集技术，相 关机构或公司就可实时得知网民的消费趋向、偏好、兴趣等等，再依有关数据推出量身定做的“商业资讯”如促销等等。

目前业界亦积极推动”物联网“(Internet of Things)工程，即把所有的居家电器用品都连接上网络，举例，目前的家居监管系统已可通过智能手机来检测或开关等，就只要连上网络就可以了。

当“物联网”成真后，大数据技术将如鱼得水，获得数据的来源大增，分析也越拉越精确。用户只要天天通过不同管道“上网”，几乎所有的“一举一动”都难逃大数据技术的掌控。其实在欧美已有许多人关注这类隐私问题，业界正致力解决之。

说有一天，大数据技术还能用来预测某人有无犯罪倾向，就像电影“关键报告”（Minority Report）里的情节一样。

分别是，“关键报告”电影里是用“神通”来预知犯罪事件，而现实生活里是用“大数据”而已。

目前已逢消费服务税季节，本地华裔中小型企业的挑战重重，笔者衷心希望“头家”们多多了解和善用诸如大数据技术般的新科技，以数据说话，为企业制定合适的策略，借以提升企业。

Oracle's SOA Applications Adapter (SOAAppsAdapter) Update Issue

Bloody Hell. This SOAAppsAdapter thingy is so bloody tedious.

So I have SOA Composites having AqAdapter as partner to listen to Business Events.

The config for this AqAdapter however is not in EM Console, instead it is in Weblogic Console - > Deployment -> OracleAppsAdapter. Yes funny thing is It is known as "OracleAppsAdapter". So confusing.

So I changed the JDBC reference inside this OracleAppsAdapter config. Next I "update" OracleAppsAdapter. However, the pending Business Events were still not picked up by the AqAdapter.

What's wrong?

Ok. We just have to restart the SOA Composite! I do not know why, perhaps the current instances were still holding the "cache" of the AqAdapter. Something like that I guess.

If you know why, appreciate your feedback in this. Thank you.

Eclipse Build, M2Eclipse and Maven Build

Maven exists for quite many years already, I think close to 8 years?

But I only started dealing with Maven few months back.

I have this Maven project in my STS(Spring Tool Suite). The project is using Maven Resource Filtering and also the project also needs to generate java web service proxy classes based on a wsdl.

So what happened is somehow the Eclipse Build will not copy the file under resources folder to the target classes folder. In the end I have to specifically run Maven Build (with goals: clean compile package) to make that works.

If I only ran Maven Clean, it will not help. Instead it will screw up the project by deleting many things from the target folder and no auto geneate.

If you know the way to have Eclipse default Build to understand Maven build process. Please let me know. Thank you.