Dev and Ops

My company's CIO recently came out with an interesting idea.

Before that, let me briefly explain the background for this.

In an organization where the core business is non-IT, the IT department usually gets vendors to help implement an IT solution or product and the vendors should do a KT to the IT department staff so that the IT department staff is able to help maintain the solution or product.

Maintenance usually means :

1) To make sure the solution or product is working well 24x7. If things go wrong, usually it is due to environment issues. For e.g. an version upgrade of one of the products that is also part of the whole solution. Sometimes it could be due to mistakes as the current solution does not read data properly under certain scenarios.

2) To enhance the product or solution to provide more values to the users. Most of the time the values means to save cost for performing one transaction. What kind of cost to save? Maybe to reduce the number of steps for the users to perform a transaction; or maybe to remove some of the manual processes such as "verify data in excel by eyes" thus avoid human error. 

Usually enhancement comes in the form of Change Request. Change Request is very common term used in the IT department of all organizations.

If it is something bigger than a "product/solution enhancement", usually such requirement will be turned into a Project. So the management or PMO will set up a project team for this.

The members of the project shall consist of Project Managers, Solution Architects, Business Analysts, and Developers, and sometimes Testers too.

Every project must be reviewed by Enterprise Architects. According to TOGAF framework, Enterprise Architects should govern the IT architecture of one organization. The value of Enterprise Architect is usually to create "Business Services" to perform certain "Business Processes", and after that the EAs have to define Information Services for each Business Service.

Anyway, my company does not have a EA team right now. As far as I know the management are working on it. When will these come into reality, I don't know.

Let's get back to Project. The problem now is there are so many issues happened to a recent project despite all the proper planning. So CIO and his team decided that:

1) To create a DEV division and a OPS division in one IT team. Take the Middleware team as the example, Middleware has one lead at the moment. In the future there will be a lead for DEV and a lead for OPS. Some developers will be parked under DEV, some developers will be parked under OPS. DEV will need to be held accountable for Project, and OPS will be held accountable for OPS.

This arrangement is not a fixed thing for developers so one developer can jump freely from DEV or OPS and vice versa at anytime.

This arrangement only matters most to the top management so that it is easier for them to check for status with the right person, in this case the lead. If some issues happened in project, look for DEV lead. If something screwed up in operation, look for OPS lead. This is how it works. Top management don't really care if the developer is in DEV or OPS.

淺談大數據

阅报得知，前公司欲发展“大数据” (Big Data) 工程，以迎合政府的“国家大数据框架”政策。

近几年来，“大数据”是信息工程界的主流。中国风云人物马云最近受访时也说，未来是“数据的时代”，笔者肯定马先生有此一说与大数据技术息息相关。

大数据之所以流行，是因为实时分析海量数据的技术已成熟，重点在于“实时”与“海量”。

换做是从前，要实时为所流入的数据作分析非常困难。往常的作法是，技术人员须写程序，天天按时把数据从不同的“数据库”(Database)集中于“数据 仓库”（Data Warehouse），才能做出所谓的商业分析报告，通常是每日报告，每星期报告，每月报告，每年报告等等，业界称之为商业智能（Business Intelligence），惟就是无法作出“实时报告”。

“大数据”另一特点是可分析来自不同管道的实时数据。比较从前的各种技术，数据须先进入“数据库”后方可作分析。而大数据技术由于可接上不同的“技术协 议”，不必等待数据进入数据库后才能做事。“网民”在网上的一切活动，如使用脸书、推特及其他社交程序，在大数据技术支援下，再配合传统数据搜集技术，相 关机构或公司就可实时得知网民的消费趋向、偏好、兴趣等等，再依有关数据推出量身定做的“商业资讯”如促销等等。

目前业界亦积极推动”物联网“(Internet of Things)工程，即把所有的居家电器用品都连接上网络，举例，目前的家居监管系统已可通过智能手机来检测或开关等，就只要连上网络就可以了。

当“物联网”成真后，大数据技术将如鱼得水，获得数据的来源大增，分析也越拉越精确。用户只要天天通过不同管道“上网”，几乎所有的“一举一动”都难逃大数据技术的掌控。其实在欧美已有许多人关注这类隐私问题，业界正致力解决之。

说有一天，大数据技术还能用来预测某人有无犯罪倾向，就像电影“关键报告”（Minority Report）里的情节一样。

分别是，“关键报告”电影里是用“神通”来预知犯罪事件，而现实生活里是用“大数据”而已。

目前已逢消费服务税季节，本地华裔中小型企业的挑战重重，笔者衷心希望“头家”们多多了解和善用诸如大数据技术般的新科技，以数据说话，为企业制定合适的策略，借以提升企业。

The importance of having a Integrated Dashboard for better IT management

 

Having an Integrated Dashboard is helpful in terms of IT Management. At least it is very very useful in terms of standalone program management according to my experience.

What kind of support we can get from the Integrated Dashboard?

1) Have a glance of how many "Interfaces" or "Services" we have. There will be a search feature to search for the Interface or Service  that we want to go to.

2) To configure Interfaces' or Services' variables at run time. For e.g. SQL, SQL Input Parameters, Database Connection, FTP Connection, Email Connection and etc.

3) To deploy latest code to the specific interface or service. Any Source Repository tool such as Mercurial, SVN, Github, and build such as Maven and Jenkin will be integrated to this Dashboard. The dashboard will provide a button to build the latest source code.

4) To provide scheduler function to schedule the launch the specific interface or service.

5) To provide server health (CPU, Memory) monitoring options using existing API such as JavaMelody or others. 

All the above are some important features that an Integrated Dashboard can have. Of course we can add more things such as monitoring , audit and etc. And of course we have to try to make sure each and every of our "interfaces" or "services" will be registered under this Dashboard. To build this Dashboard from scratch is going to take tremendous effort.

My thought on future IT Work Process

Nowadays in a lot of organizations, we have the "Application Team", "Middleware or SOA or ESB Team", we have the "Infrastucture Team", "DBA Team" and so on.

Imagine a project needs to involve so many different teams just to implement a solution. I would not say this working model is bad or not efficient or whatever negative term you can think of. Because so many IT solutions were implemented this way. The thing is, however, usually when working with different teams, communication becomes very very important and we've got to deal with a lot of human factors due to this. Human factor is more difficult to deal with than any technical problem.

Many times a team needs to depend on another team to complete the necessary task first. Such dependency creates a lot of troubles when the dependent team works in silo and does not respond in time using different kind of excuses. For e.g. if the Development team wants to increase server memory or to request to open certain port or things like that, in this case the Development team has to approach the Server team and Network team, and this may end up requires the Development team to fill in dozens of form, to attend to dozens of meeting to provide justification and etc. From the business perspective, this is actually not "acceptable", you know, every minute counts in the business world.

So I'm thinking, why not we have "development unit" only, or we remain only the "application team", then we remove the Middleware team, DBA team, Server team and Network team. Each application team will have their own Middleware, DBA, Server Admin and Network Admin.  And we'll set up the "Architect Team" to govern the Middleware, DBA, Server and Network standards. The Architect Team will consist of Enterprise Architect, Application Architect, Middleware Architect, DBA Architect, Server Admin Architect and Network Architect to coordinate with each other. They are the one who set the standards to the whole organization and each projects, they have to provide the architectural details for each project and review the architecture constantly at different project phases.

Project operation wise, architects should be not involved at all. Project manager will have full power on his/her team member consisting of BAs, Developers, System Admins, Network Admins, DBAs and etc. In that case we can remove the "form filling bureaucracy" and improve the project speed.

I'm not sure if any organization out there is already doing this way.