The importance of having a Integrated Dashboard for better IT management

 

Having an Integrated Dashboard is helpful in terms of IT Management. At least it is very very useful in terms of standalone program management according to my experience.

What kind of support we can get from the Integrated Dashboard?

1) Have a glance of how many "Interfaces" or "Services" we have. There will be a search feature to search for the Interface or Service  that we want to go to.

2) To configure Interfaces' or Services' variables at run time. For e.g. SQL, SQL Input Parameters, Database Connection, FTP Connection, Email Connection and etc.

3) To deploy latest code to the specific interface or service. Any Source Repository tool such as Mercurial, SVN, Github, and build such as Maven and Jenkin will be integrated to this Dashboard. The dashboard will provide a button to build the latest source code.

4) To provide scheduler function to schedule the launch the specific interface or service.

5) To provide server health (CPU, Memory) monitoring options using existing API such as JavaMelody or others. 

All the above are some important features that an Integrated Dashboard can have. Of course we can add more things such as monitoring , audit and etc. And of course we have to try to make sure each and every of our "interfaces" or "services" will be registered under this Dashboard. To build this Dashboard from scratch is going to take tremendous effort.

My thought on future IT Work Process

Nowadays in a lot of organizations, we have the "Application Team", "Middleware or SOA or ESB Team", we have the "Infrastucture Team", "DBA Team" and so on.

Imagine a project needs to involve so many different teams just to implement a solution. I would not say this working model is bad or not efficient or whatever negative term you can think of. Because so many IT solutions were implemented this way. The thing is, however, usually when working with different teams, communication becomes very very important and we've got to deal with a lot of human factors due to this. Human factor is more difficult to deal with than any technical problem.

Many times a team needs to depend on another team to complete the necessary task first. Such dependency creates a lot of troubles when the dependent team works in silo and does not respond in time using different kind of excuses. For e.g. if the Development team wants to increase server memory or to request to open certain port or things like that, in this case the Development team has to approach the Server team and Network team, and this may end up requires the Development team to fill in dozens of form, to attend to dozens of meeting to provide justification and etc. From the business perspective, this is actually not "acceptable", you know, every minute counts in the business world.

So I'm thinking, why not we have "development unit" only, or we remain only the "application team", then we remove the Middleware team, DBA team, Server team and Network team. Each application team will have their own Middleware, DBA, Server Admin and Network Admin.  And we'll set up the "Architect Team" to govern the Middleware, DBA, Server and Network standards. The Architect Team will consist of Enterprise Architect, Application Architect, Middleware Architect, DBA Architect, Server Admin Architect and Network Architect to coordinate with each other. They are the one who set the standards to the whole organization and each projects, they have to provide the architectural details for each project and review the architecture constantly at different project phases.

Project operation wise, architects should be not involved at all. Project manager will have full power on his/her team member consisting of BAs, Developers, System Admins, Network Admins, DBAs and etc. In that case we can remove the "form filling bureaucracy" and improve the project speed.

I'm not sure if any organization out there is already doing this way.

Old school Axis 1.0 way of consuming Web Service :(

Ok. Although nowdays we have a lot of advanced web service libraries such as CFX and etc, we still have to admit that sometimes we do bump into situation that we have no choice but to use the grandpa way of consuming web service! (well at least we are lucky not to use the great-grandpa way. LOL)

Yeah somehow I have to use the Axis 1.x library to generate Web Service Java Clients or Stub for me. Well that is not a big problem but but but in my case the Web Service Provider needs SOAP Header which contains all the username token information.

That's a real headache ya!

Well, the solution is quite easy anyway. It looks like below:

import org.apache.axis.client.Stub;
import org.apache.axis.message.SOAPHeaderElement;

SOAPHeaderElement security = new SOAPHeaderElement("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd","Security")

SOAPHeaderElement usernameToken = new SOAPHeaderElement("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd","UsernameToken");

SOAPHeaderElement user = new SOAPHeaderElement("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd","Username", "xxx"); 

SOAPHeaderElement password = new SOAPHeaderElement("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd","Password", "xxxyyy");
usernameToken.addChild(user);
usernameToken.addChild(password);
security.addChild(usernameToken);
            
Execute_ptt exec = ws.getexecute_pt(new URL("http://localhost:8001/soa-infra/services/SampleWebServiceProvider"));
((Stub)exec).setHeader(security);

Next, just make your web service call using the stub.

Fixed Length File Without Delimiter and BeanIO

What happened is I have this IBM Mainframe's CobolCopyBook and I need to read a number of fields from there.

For your information the CobolCopyBook is a FixedLength file which has NO Delimiter.

My problem is that in my environment I have to stick to the BeanIO framework for the parsing. That means I cannot use the normal BufferedReader + substring way which is considered NOT SO ELEGANT way to solve this problem.

Also this is the first time I am using BeanIO. I found that the existing FixedLength utilities in BeanIO, such as the FixedLengthReader expects each line has a Carriage Return (CR or \r) or Line Feed (LF or \n). But it also allows the user to specify a "delimiter" to determine when to assume a Line.

Unfortunately in my use case, as I mentioned earlier , my file does not have any delimiter. Basically it is one string of characters. Worst still is one of the field always has the CR or LF character in it! What will happen is the BeanIO's reader will consider whatever after the CR or LF character is a brand new Line but the fact it is NOT!

To summarize, I have 2 problems here.
1) No delimiter to determine a new line
2) One of the field contains Carriage Return and Line Feed 

I was cracking head for these two problems and my colleague came out with some brilliant ideas. They suggested me to use OuterBean which contains the Target Bean as a Segment. Segment as one annotation is another BeanIO feature that does all the grouping or nested bean stuff. This approach will solve the "No Delimiter to break line" issue.

Another colleague suggested to create a custom java.io.Reader that extends from java.io.Reader and implements my own "read()" method. Because in the end BeanIO is going to invoke the "read()" method from the Reader object that is passed into its internal classes. So inside this read() method I will check if the character is either a Carriage Return or a Line Feed then I will return a space means I am actually replacing \n and \r with space. This will solve my second problem where the Carriage Return or Line Feed will cause BeanReader to take the characters after it as a new line.

So now I managed to fix these two problems with the OuterBean approach and Customized java.io.Readar approach. However this is not so complete because I don't like the replacing carriage return and line feed character with space.

public class CustomerOuterUserBean {

    @Segment(name="customerUserBeans", collection=ArrayList.class, minOccurs=0, maxOccurs=-1, type=CustomerUserBean.class )

    private List customerUserBeans = new ArrayList();

public class NpiFilterReader extends FilterReader {

    public NpiFilterReader(Reader in) {
        super(in);
    }

    @Override
    public int read() throws IOException {
        int read = super.read();
        if (read == '\r' || read == '\n')
            return ' ';

        return read;
    }

So I continued the journey and in the end I found a very very simple solution for these two issues.

What I did is to create a custom RecordParserFactory which manipulates a custom RecordReader and I overwrite the read() method for the custom RecordReadear. The read() method in the custom RecordReader  looks like follows:

public class MyFixedLengthRecordReader extends FixedLengthReader

    public String read() throws IOException, RecordIOException {

        char[] buffer = new char[300];

        if(in.read(buffer, 0, 300) != -1){

            return new String(buffer);

        }      

        return null;

    }

Note that in my use case my line is fixed at 300 characters per line.

This is the most elegant yet simple solution! Thank Divine!