Why Server Appliances Are Valid

I would like to argue a valid option (NOT the only option) to build appliances based upon their intended functionalities. All the key software will be installed into the appliance.

We can build an Operational/Transitional Appliance, which can contain application server (such as WebSphere Application Server, JBoss, IIS/.NET Framework), database (such as Oracle, SQLServer) and BPM server (such as WebSphere Process Server, jBPM, BizTalk Server), Business Rule Management server (IBM ILog JRules, JBoss Rule, BizTalk BRE), Data Service server (PowerCenter, SQL Server Integration Services, Talend) and others in one physical box.

We can build BI/Analytical Appliance, which can contain BI server (BusinessObjects, Jasper), database (such as Oracle, SQLServer), Business Rule Management server (IBM ILog JRules, JBoss Rule, BizTalk BRE), Data Service server (PowerCenter, SQL Server Integration Services, Talend) and others in one physical box.

This appliance can also be used to run big batch job with the following added into the mix: Job/Workload Management server (CA Autosys, Quartz Job Scheduler) and large volume data facilitator (PowerCenter Partitioning Option. SSIS Pipeline, Talend MPx).

Yes, there a lot of things going on within this one box. But there are definitely several advantages: a). it is much easier to manage; b). due the eliminated network latency, in some cases, it might perform better than putting each software server category into their own boxes – a popular topology during last decade; c) we can still have load-balancing and failover for the appliance; d) we can isolate the software servers using virtualization, which will increase fault tolerance and capacity allocations; e). if something goes wrong beyond repairing, simply throw the appliance away and put another one in; f) last but not least, it helps break the barriers among operational/transactional, BI/analytical and batch by sharing the metadata and business rules.

My Architecture Practices

• Setting up the right boundaries and govern them are among the most important architecture works. Here is how I set the boundaries: organize the code that is based upon Data Service, Business Domain Service, Business Rule Service, Business Process Service, Integration Service, Security Service, Presentation Channel Service, and Common Technical Service, NOT, again NOT, based upon projects. It is forbidden to put project names in packages, classes, and codebase folders.
• Establish the core: Business Architecture, Data Architecture and Solution Architecture – technical; delivery and vendor management – administrative; explore the rest
• Business Architecture belongs to the business, dotted line to IT. Work with business people on daily basis, not project basis
• Throw away the word document; it is evil. Use Enterprise Social software to create architecture repositories and facilitate knowledge accumulation and knowledge sharing
• Break the barriers among transactional, operational, analytical, and batch applications. Data Service Hub and metadata hold the key to make it happen.
• Emphasize the creation of foundations: business vocabulary and metadata
• Establish the Center of Excellence: every project goes here and is filtered by it and governed by it, NO exception
• Big picture guided Agile: Big Pig Agile Methodology
• Two work stream: project and sideline. Change the budget and funding structure to support sideline work stream
• Evaluate project based upon: efficiency, effectiveness, predictability, and consistency. Not just finish the project. Have third party evaluate the project objectively.
• Separate the code base of quick and dirty projects from long shelf life projects
• From technical sequencing perspective, make sure that Data Integration/MDM and security/identity management architectures are on the solid ground, before starting tackling integrations such as SOA (although SOA as a whole is beyond just architecture) and Portal

Funny Project Stories - part 1

When somebody promotes better ways of doing things, some people in project often ask, “why this one? why right now?”

“Time to market” is the most used excuse – “we have something to deliver”; the other one is “we are so unique that the new ways don’t apply here”.

The answer is simple, “If we don’t start somewhere, we will end up in nowhere.” because the following situations will surely happen:
1. “a penny wise but a dollar foolish”;
2. “Yesterday's solutions are today's problems”

Funny Story 1:
Hire an expensive integration developer, but only give him a machine with 1 GB of memory and a weak CPU. So his machine crawls during debugging code in WebSphere Integration Developer.

Funny Story 2:
Create a small database for almost every project which ends up having many of them after a few years. Then spend millions of dollars if not more, often unsuccessfully to integrate these scattered and unruly databases when business needs to have data from all of them.

Application Level Software vs. Enterprise Level Software

While the left side should be valued; the right side should
be added to reach to higher level of maturity and realize the
efficiency, effectiveness, predictability and consistency of every project.

Application Level Software	Enterprise Level Software
Business rules are being internalized and imbed in the code	Business rules are externalized and can be deployed to and shared by multiple application building blocks
Emphasize implementability	Emphasize extensibility and reusability
Solve the problems at hands within a project	Solve the problems at hands within the context of enterprise
Often modulize code at function,class, and component levels	Modulize code to reach to service and building blocks levels
Tend to not to respect the boundaries	Follow the service and building blocks boundaries
Only the project related domainas it domain background	The entire business segment as its domain background
Focus on code syntax optimization	Focus on service/component boundary setting
Pieces on hands are the big picture	Emphasize how smaller pieces fit into the big picture
Often treat every project as its own	Treat every project as an integration project
Loosely following industry standards such as LDAP forsecurity repository	Empasize using industry standards
The barriers among transactional, operational, integrational,analytical, and batch modules are high	The barriers among transactional, operational, integrational, analytical, and batch modules are broken
Often use the project name as boundary setter, for example, com.companyname.projectname	Use business architecture blocks as the boundary setter, for example, com.companyname. businesssegmentname. businessrule
Skillset tend to be generic	The right mixture of specialists and generalists at building blocks level
Often take a project view	Take portfolio view
Often dive into nitty-gritty line of code optimization	Emphasize the code organization
Starting from project specific requirement	Starting from business architecture
The lines among business rules, business processes, business events, business data, business contents, business intelligence and etc. are blurred	The lines among those are intentionally defined and well integrated using controlled ways
On time delivery, under budget and work as being intended with less bugs for that project are main successful indicators	Scalable, extensible and evolvable with ease and costing less are added

Enterprise Social Software as an IT Project Platform

There is no doubt that enterprise custom application software is a type of social activities: people with different skills, experiences, work habits, even personalities work together to deliver something that can add values to business stakeholders.

The collaborative nature of IT project is more paramount when team members come from different locations (inshore and offshore), systems, and departments.

What the enterprise social software has tried to solve is exactly that: collaboration, although in a much broader sense. Here is the definition, enterprise social software is, “a system of web-based technologies that provide rapid and agile collaboration, information sharing, emergence and integration capabilities in the extended enterprise.” (http://www.aiim.org/What-is-Web-2.0.aspx).

Andrew McAfee listed 6 functionalities of enterprise social software (http://sloanreview.mit.edu/the-magazine/articles/2006/spring/47306/enterprise-the-dawn-of-emergent-collaboration/):

• Search: allowing users to search for other users or content
• Links: grouping similar users or content together
• Authoring: including blogs and wikis
• Tags: allowing users to tag content
• Extensions: recommendations of users; or content based on profile
• Signals: allowing people to subscribe to users or content with RSS feeds

Dion Hinchcliffe added 4 more:

• Freeform function: no barriers to authorship (meaning free from a learning curve or from restrictions)
• Network-oriented function, requiring web-addressable content in all cases
• Social function: stressing transparency (to access), diversity (in content and community members) and openness (to structure)
• Emergence function: requiring the provision of approaches that detect and leverage the collective wisdom of the community

All of these 10 are completely relevant to the entire enterprise software life cycle. I especially value the “no barriers to authorship” feature because it can greatly expedite the knowledge accumulation and sharing. I have always advocated the work stream that is independent of project stream to prepare for things needed for projects. For instance, it will be almost infeasible for business to sponsor a project called Business Vocabulary project. There are no direct benefits seen from their angles. However, Business Vocabulary is a key foundation for any projects. We often find out that all kinds of business vocabularies being embed at the end of long requirement document repeatedly and inconsistently. With the enterprise social software, we can encourage everybody, I mean everybody, especially business people to put their understanding on a definition of a piece of vocabulary there for others to criticize and give feedbacks. In a few years, there will be complete and accurate business vocabulary repository. People who ever contribute to it will see the benefits and take pride of it: team work, right?

Many of software tools must incorporate features of enterprise social software right now or in the near future. Otherwise, they will become irrelevant. For instance, IBM Rational RequisitePro 7.0 enables “anyone with Web access, independent of his or her platform, can view, author and manage requirements quickly and efficiently without having Rational RequisitePro loaded on their machine. Project administration can be done via the web too.”

The desirable scenario is to conduct software project from project management, requirement, architecture, design, code publishing, and quality control entirely on the web using enterprise social software as the platform. The RIA (Rich Internet Application) technologies such as AJAX, Flash and JavaFX will speed it up for sure.

Since everybody can see how Google Docs works, I will use it as an simple example. The word description, UML diagram and tables flow and mesh up naturally in Google docs although primitively, although the diagram drawing part must be enhanced significantly to draw all types of UML diagram. It should also include diagram organizations that only include folders holding diagrams. Nevertheless, it represents the right direction for utilizing enterprise social software. A business analyst can write requirement using word features and draw UML diagrams. He can then publish his document on the web. All stakeholders can see it immediately and more importantly provide feedbacks which can be appended to the document. Or they can modify the document directly. The document comparison feature will allow interested parties to trace the changes people have made all along. The involved parties can also tag the document, which enables more efficient searches.

Enterprise social software is a key enabler of IT project. Let’s use it to unleash the power of the true collaboration.

The Power of Zachman Framework

More often than not, I like things that are natural and simple. The beauty of Zackman Framework is just that: natural and simple. In these years as an architect, I have referred back to it many many times, especially during early years. Now it has kind of printed into my brain with the vivid picture and molded my thinking processes when conducting architecture and design.

Zackman Framework has two dimensions out of box. The horizontal dimension uses the 5W+H (What, Where, When, Why, Who and How). I still remember at the first composition class in primary school, the teacher told us how to use 5W+H to tell a complete story. It helped. It is surprising that this rudimentary stuff can apply to the IT architecture and can do it so well.

The vertical dimension is harder to understand than horizontal one because it involves the level of details and changes of perspectives from top to bottom. I have seen the struggles even among experienced IT professionals to be able to peel the different layers of details either from inside-out or from outside-in. And yet this is one of the most essential skills especially for IT architects. The vertical dimension is top-down in nature, which means when it flows from Contextual, Conceptual, Logical, to the Functioning Enterprise, the pictures get narrowed and more details revealed. One of most striking powers of this layering is that how clearly and naturally it switches from the business concerns/perspectives to IT perspectives. People like to talk about business and IT alignments, often end up paying lip services due to the lack of grasping how the information/knowledge flows from business to IT. This vertical dimension can definitely help.

The intersections of the rows from horizontal dimensions and columns from vertical dimension form 36 cells, which cover the most complete and inclusive concerns among all the popular enterprise architecture frameworks. People can easily find homes for the concerns from other frameworks.

For instance, for the famous 4+1 View framework, its Logical view can be related to the third row – System Model; and yet the later is richer because it goes further as to 5W+H. Its Process View can be related mainly second and third cells of “How” vertical dimension. Its Physical View can be related to the first three cells of Technology Model. Its Development View can be related to row Detail Representations. Its Use Case/Scenarios View can be related to middle cells of Business Model row. The more powers go to the fact that the neighbor cells of each cell serve as its context, so people will never lose sight of where they are when addressing the specific concerns.

Another well-known framework, The Open Group Architecture Framework (TOGAF), its four architecture categories can also easily be mapped to the Zackman Framework: the Business Architecture can be mapped to the first two rows of Zachman; the Data Architecture can be mapped to cells in the row “What”; the Application Architecture can be mapped to the row of System Model; the Technology Architecture can be mapped to the row of Technology Model.

One of main complaints from IT practitioners is that the Zackman Framework tends to be documentation heavy: it sounds scary that 36 cells will correspond to 36 documents if each cell just generates one document. Making 36 documents consistent and cohesive across time span will be extremely costly in larger enterprise, not even mentioning to absorb them by team members. I have found this complaint almost groundless. First, due to its top-down layering nature, there will be much less documents in first two rows. The content of these documents will get enriched when moving down these rows. Secondly, I have proposed so called “two work streams” approach (will talk more in different blog entry): project stream and support stream. The support stream will largely handle the document repository for many cells. Enterprise social software (aka Enterprise 2.0) can be used to expedite the creations and consumptions of these documents.

The Zackman Framework is more of taxonomy of enterprise IT than of a methodology, which means it does not address how people will arrive at the artifacts that these 36 cells will need to produce.

I have long promoted the so called “big picture guided” software development to supplement and remedy the weakness of Agile methodology (will talk more in different blog entry). Regardless to their roles and positions in IT, every team members should have certain levels of understanding about the big picture. Their angles and depth of this knowledge might vary depending on their interests. This belief is largely motivated by the Zackman Framework. Its first two rows (Contextual and Conceptual) will form the solid big picture for any IT practitioners. My Layered Painting SDLC is a great match to implement the Zachman Framework.

I strongly believe that every IT practitioner should know the Zachman Framework. Each profession has its own way of thinking and behaviors, which is molded by training and long time practices. The Zachman Framework is a very powerful mean to mold an IT practitioner into a true professional.

Business Architecture – a Further Explanation

At the business segment/unit level, more than anything else, these three technical areas are at the core of any IT shops if they want to conduct IT projects consistently, predictably, effectively and efficiently (every word counts here). They are: business architecture, data architecture, and solution architecture. The rest is supplemental. Without knowing what they are and how to utilize their power and potentials, an IT shop will not be successful, period, end of the story, “Yes, Sir”. People may score now and then, but they will not always score.

Among these three, business architecture has caused most confusion and has been recognized the least. And yet, it is THE most critical one.

When you talk to business people about business architecture, especially those who are in charge of business strategy, they often stare at you and make you feel you are talking some nonsense, if not hurt their feelings. They might inform you that they ARE the business architects. How come there is different business architecture from IT side? You are stepping on my toes.

Some of IT folks may also feel the same way. If examining the texts about business architecture, we must admit that the confusion is almost a self-inflicted wound, because even some IT architecture experts are having troubles to explain the business architecture and the differences between business architecture on the business side and the business architecture on the IT side.

Here is one of the most concise and yet thorough definition about business architecture from The Object Management Group – Business Architecture Special Interest Group (BASIG), since it is not that long, I will paste here for the sake of completeness,

“

Business Architecture Overview

Business Architecture defines the structure of the enterprise in terms of its governance structure, business processes, and business information. In defining the structure of the enterprise, business architecture considers customers, finances, and the ever-changing market to align strategic goals and objectives with decisions regarding products and services; partners and suppliers; organization; capabilities; and key initiatives.

Business Architecture primarily focuses on the business motivations, business operations and business analysis frameworks and related networks that link these aspects of the enterprise together.

In order to develop an integrated view of an enterprise, many different views of an organization are typically developed. The key views of the enterprise within the business architecture are: 1) the Business Strategy view, 2) the Business Capabilities view, 3) the Business Process view, 4) the Business Knowledge view, and 5) the Organizational view.

The Business Strategy view captures the tactical and strategic goals that drive an organization forward. The goals are decomposed into various tactical approaches for achieving these goals and for providing traceability through the organization. These tactical and strategic goals are mapped to metrics that provide ongoing evaluation of how successfully the organization is achieving its goals.

The Business Capabilities view describes the primary business functions of an enterprise and the pieces of the organization that perform those functions. This view further distinguishes between customer-facing functions, supplier-related functions, business execution, and business management functions.

The Business Process view defines the set of strategic, core and support processes that transcend functional and organizational boundaries. It sets the context of the enterprise by identifying and describing external entities such as customers, suppliers, and external systems that interact with the business. The processes also describe which people, resources and controls are involved in the process. The lowest process level describes the manual and automated tasks that make up workflow.
The Business Knowledge view establishes the shared semantics (e.g., customer, order, and supplier) within an organization and relationships between those semantics (e.g., customer name, order date, supplier name). These semantics form the vocabulary that the organization relies upon to communicate and structure the understanding of the areas they operate within.

The Organizational view captures the relationships among roles, capabilities and business units, the decomposition of those business units into subunits, and the internal or external management of those units.

In addition to the above views of the enterprise, the relationships connecting the aforementioned views form the foundation of the business architecture. This foundation provides the framework that supports the achievement of key goals; planning and execution of various business scenarios; and delivery of bottom line business value.

”

I must say this is excellent stuff. What it tries to say is that business architecture takes holistic view on the enterprise as a whole and business segment as the part of the whole. Under it, there are five sub-views, which the business architecture will integrate them together. This definition addresses the first two rows of Zachman Framework: Contextual and Conceptual. To me, it still does not go far enough to differentiate business architecture on the business side and business architecture on the IT side. With this solid foundation, here is my further explanation:

1. Business architecture is the most critical bridge between business and IT. A business architect must not only have intimate knowledge and deep understanding about the business segment he is in like what the above describes, but also must understand how IT functions and how IT people such as other types architects think and behave. The business architect must be willing and feel comfortable to closely work with IT people. These are the first layer of divergence between business architecture on the business side and business architecture on the IT side: they have different key audiences.

2. The knowledge base and artifacts of business architecture must be easily consumable by IT people, which include but is not limited to: the structure of the content and the presentation style of the content. Therefore, it is absolutely essential for a business architect to have strong modeling (UML, BPMN) skills and story telling skills. The foundation of the knowledge base and artifacts is the business vocabulary repository, as the foundation of data architecture is metadata. The business vocabulary repository must be again independent of each on-gong project. A business architect should harvest the new vocabularies from each new project, at the same time feed and reuse the existing vocabularies to each new project, hence to form what I have called the circle of “Plant and Harvest”. This practice also applies to other three important repositories: business domain repository, business rule repository and business process repository. This is the second layer of divergence between business architecture on the business side and business architecture on the IT side: they produce and maintain different knowledge base and artifacts.

3. The business architecture will serve the IT projects from envision, solution, implementation, all the way through operation. So it deals with the entire SDLC process. Business architecture on the business side most likely will stop at the envision stage. This is the third layer of divergence between business architecture on the business side ad business architecture on the IT side: they engage in different stages of SDLC. So if we look at the Zackman Framework again, the business architecture is everywhere on the 36 cells.

I hope this short article can raise the awareness of the criticality of business architecture, and push the IT shops to formally setup the role of business architect and gradually utilize its powers. In large organizations, there might be two or more levels business architecture: enterprise level and business segment level. An enterprise level business architect without business segment level business architect will be easily getting shallowed because nobody has brain capacity to tackle so many areas of the business for a large enterprise. An enterprise level business architect should be more of harvester and synthesizer from divisional level business architect and a best practice promoter and governor.

Some pioneering originations have already put business architecture under the line of business and only dotted line to the IT, which is an excellent move. In the near future, more and more organizations will follow this course.

I will write more detailed stuff in the future around the topics of business architecture.