What are the Downsides of a Microservice based solution

A Microservice based solution has the following downsides:

  1. Distributing the application adds complexity for developers when they are designing and building the services and in testing and exception handling. It also adds latency to the system.
  2. Without a Microservice-oriented infrastructure, An application that has dozens of Microservices types and needs high scalability means a high degree of deployment complexity for IT operations and management.
  3. Atomic transactions between multiple Microservices usually are not possible. The business requirements must embrace eventual consistency between multiple Microservices.
  4. Increased global resource needs (total memory, drives, and network resources for all the servers or hosts). The higher degree of granularity and distributed services requires more global resources. However, given the low cost of resources in general and the benefit of being able to scale out just certain areas of the application compared to long-term costs when evolving monolithic applications, the increased use of resources is usually a good tradeoff for large, long-term applications.
  5. When the application is large, with dozens of Microservices, there are challenges and limitations if the application requires direct client-to-Microservice communications. When designing and building a complex application based on Microservices, you might consider the use of multiple API Gateways instead of the simpler direct client‑to‑Microservice communication approach.
  6. Deciding how to partition an end-to-end application into multiple Microservices is challenging. As You need to identify areas of the application that are decoupled from the other areas and that have a low number of hard dependencies. Ideally, each service should have only a small set of responsibilities. This is like the single responsibility principle (SRP) applied to classes, which states that a class should only have one reason to change. 

 

 

Advertisements

Practical REST API Design, implementation and Richardson Maturity Model

Richardson Maturity Model classifies REST API maturity as follows

  • Level Zero: These services have a single URI and use a single HTTP method (typically POST). For example, most Web Services (WS-*)-based services use a single URI to identify an endpoint, and HTTP POST to transfer SOAP-based payloads, effectively ignoring the rest of the HTTP verbs. Similarly, XML-RPC based services which send data as Plain Old XML (POX). These are the most primitive way of building SOA applications with a single POST method and using XML to communicate between services.
  • Level One: These services employ many URIs but only a single HTTP verb – generally HTTP POST. They give each individual resource in their universe a URI. Every resource is separately identified by a unique URI – and that makes them better than level zero.
  • Level Two: Level two services host numerous URI-addressable resources. Such services support several of the HTTP verbs on each exposed resource – Create, Read, Update and Delete (CRUD) services. Here the state of resources, typically representing business entities, can be manipulated over the network. Here service designer expects people to put some effort into mastering the APIs – generally by reading the supplied documentation. Level 2 is the good use-case of REST principles, which advocate using different verbs based on the HTTP request methods and the system can have multiple resources.
  • Level Three: Level three of maturity makes use of URIs and HTTP and HATEOAS. This is the most mature level of Richardson’s model which encourages easy discoverability and makes it easy for the responses to be self-explanatory by using HATEOAS. The service leads consumers through a trail of resources, causing application state transitions as a result.

Where HATEOAS (Hypermedia as the Engine of Application State) is a constraint of the REST application architecture that keeps the RESTful style architecture unique from most other network application architectures. The term “hypermedia” refers to any content that contains links to other forms of media such as images, movies, and text. This architectural style lets you use hypermedia links in the response contents so that the client can dynamically navigate to the appropriate resource by traversing the hypermedia links. This is conceptually the same as a web user navigating through web pages by clicking the appropriate hyperlinks to achieve a final goal. Like a human’s interaction with a website, a REST client hits an initial API URI and uses the server-provided links to dynamically discover available actions and access the resources it needs. The client need not have prior knowledge of the service or the different steps involved in a workflow. Additionally, the clients no longer have to hard code the URI structures for different resources. This allows the server to make URI changes as the API evolves without breaking the clients.

Naturally you would want to build to highest standard and provide level three REST API. That would mean providing a links field as in the following example below from GT-IDStorm API

The data payload as you can see from this sample is huge, compared to the actual data returned.

{

“value”: [

{

“id”: “63b2c70e-2bcb-4335-9961-3d14be642163”,

“name”: “Entity-1”,

“description”: “Testing Entity 1”,

“links”: [

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163”,

“rel”: “self”,

“method”: “GET”

},

{

“href”: null,

“rel”: “get_entitydefinition_byname”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/full”,

“rel”: “get_full_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163”,

“rel”: “delete_entitydefinition”,

“method”: “DELETE”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/attributes”,

“rel”: “create_attribute_for_entitydefinition”,

“method”: “POST”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/attributes”,

“rel”: “get_attributes_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/systems”,

“rel”: “create_system_for_entitydefinition”,

“method”: “POST”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/systems”,

“rel”: “get_system_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/data”,

“rel”: “get_data_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/63b2c70e-2bcb-4335-9961-3d14be642163/data/GetEntityDataWithMissingSystems”,

“rel”: “get_data_WithMissingSystems_for_entitydefinition”,

“method”: “GET”

}

]

},

{

“id”: “54bc1f18-0fd5-43dd-9309-4d8659e3aa91”,

“name”: “Entity-10”,

“description”: “Testing Entity 10”,

“links”: [

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91”,

“rel”: “self”,

“method”: “GET”

},

{

“href”: null,

“rel”: “get_entitydefinition_byname”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/full”,

“rel”: “get_full_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91”,

“rel”: “delete_entitydefinition”,

“method”: “DELETE”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/attributes”,

“rel”: “create_attribute_for_entitydefinition”,

“method”: “POST”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/attributes”,

“rel”: “get_attributes_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/systems”,

“rel”: “create_system_for_entitydefinition”,

“method”: “POST”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/systems”,

“rel”: “get_system_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/data”,

“rel”: “get_data_for_entitydefinition”,

“method”: “GET”

},

{

“href”: “https://localhost:44379/api/v1/entity/54bc1f18-0fd5-43dd-9309-4d8659e3aa91/data/GetEntityDataWithMissingSystems”,

“rel”: “get_data_WithMissingSystems_for_entitydefinition”,

“method”: “GET”

}

]

}

],

“links”: [

{

“href”: “https://localhost:44379/api/v1/entity?orderBy=Name&searchQuery=Testing%20Entity%201&pageNumber=1&pageSize=10”,

“rel”: “self”,

“method”: “GET”

}

]

}

For example if we remove the HATEOAS requirement that data returned for the same query would be

This would less data would have huge impact on the system as a whole performance, Less traffic on the network, less data to process and manipulate by the client and servers.

[

{

“id”: “63b2c70e-2bcb-4335-9961-3d14be642163”,

“name”: “Entity-1”,

“description”: “Testing Entity 1”

},

{

“id”: “54bc1f18-0fd5-43dd-9309-4d8659e3aa91”,

“name”: “Entity-10”,

“description”: “Testing Entity 10”

}

]

I usually implement the API to have and accept header with multiple options

  • Application/json: returns just the data
  • Application/hateoas+json: return the data with the hateoas (Links) data.

I also implement another resource or operation that provides the links structures

In Conclusion

I would recommend implementing the API to:

  • Support Level Two and Leve Three at the same time by using the accept header for the request to
    • Application/json: returns just the data
    • Application/hateoas+json: return the data with the hateoas (Links) data.
  • Implement another resource or the root that would return the URLs (structures and operations) that are supported by the API.

As I found just support HATEAOS only would make the system pay heavy price on performance specially with large data loads while very few clients if any would utilize the links returned. I would love to hear your thoughts and experience on APIs with HATEAOS?

What is Solution Architecture,

In my opinion, Solution Architecture is defining the design or organization of different systems and components to fulfill a business needs and requirements. By systems and components, I mean software applications, and the infrastructure (hardware devices, virtual machines, on-premises or on the cloud) the software runs on. Solution architecture is the combination of Application Architecture, Integration Architecture and Infrastructure Architecture. Solution Architecture takes a high-level description of all Applications, Integrations and Infrastructure architectures in an implementation. For example, with any new application implementation in the enterprise, that new system needs definitions of:

  • Application architecture, that is what components of the application would be utilized and how they will be used by the organization, which components would need customizations and how the system would be configured to accomplish the goals the business wants.
  • Integration with other systems. The new application at least would need to connect to the enterprise Identity management store and how to provision and manage users access to the system and export information to the enterprise data warehouse or bigdata system such as Hadoop or analytics systems, and diagnostics and monitoring to the enterprise monitoring system.
  • Infrastructure, that is an environment to run on wither on premises or in the cloud, that is servers, operating systems, networks, firewall, storage, security and access control, auditing etc.

As solution architect will need to understand the business requirements, the enterprise IT standards, and work on the selection of the appropriate software application to satisfy the business requirements, the enterprise standards, project budgets, and author a solution blueprint that describes how the proposed architecture. Solution architecture is tightly coupled with Enterprise Architecture and infrastructure Architecture teams in most organizations. Solution architecture is one of the key methods, by which enterprise architecture delivers value to the organization. Solution architecture activities take place during solution ideation, solution design, and solution implementation. During ideation, solution architecture establishes the complete business context for the solution and defines the vision and requirements for the solution. During design, solution architecture elaborates potential options, which may include RFIs, RFPs or prototype development. It selects the most optimal option and develops the roadmap for the selected solution. During implementation, solution architecture communicates the architecture to the stakeholders, and guides the implementation team.

Typically, a Solution Architect will work project managers on defining tasks, estimates, discusses architecturally significant requirements with stakeholders, designs a solution architecture blueprint, evaluate proposals, communicates with designers and stakeholders, document solution alternatives. There are four core activities in solution architecture design.

  1. Architectural analysis is the process of understanding the environment in which a proposed solution will operate and determining the requirements for the solution. The input or requirements to the analysis activity can include items such as:
    1. what the system will do when operational (the functional requirements)
    2. how well the system will perform runtime non-functional requirements such as reliability, operability, performance efficiency, security, compatibility.
    3. development-time non-functional requirements such as maintainability and transferability
    4. business requirements and environmental contexts of a system that may change over time, such as legal, social, financial, competitive, and technology concerns
    5. Organizational development capabilities
  2. Architectural synthesis or design is the process of creating an architecture. Given the architecturally significant requirements determined by the analysis, the current state of the design and the results of any evaluation activities, the design is created and improved.
  3. Architecture evaluation is the process of determining how well the current design or a portion of it satisfies the requirements derived during analysis. An evaluation can occur whenever an architect is considering a design decision, it can occur after some portion of the design has been completed, it can occur after the final design has been completed or it can occur after the system has been constructed.
  4. Knowledge management and communication is the activity of exploring and managing knowledge that is essential to designing a solution architecture. A solution architect does not work in isolation. SA get inputs, functional and non-functional requirements and design contexts, from various stakeholders; and provides outputs to stakeholders. Solution architecture knowledge management activity is about finding, communicating, and retaining knowledge. As software architecture design issues are intricate and interdependent, a knowledge gap in design reasoning can lead to incorrect software architecture design.
  5. Documentation is the activity of recording the design generated during the solution architecture process. A solution architecture is described in a solution blueprint that usually contains
    1. Business Architecture section: The business strategy, governance, organization, and key business processes
    2. Interface Architecture (APIs): A blueprint for the individual application interfaces both User Interface and Application Programming Interface and their relationships to the core business processes of the organization.
    3. Application Architecture section: A blueprint for the individual application components to be deployed, their interactions, and their relationships to the core business processes of the organization.
    4. Data Architecture Section: The structure of the application logical and physical data assets and data management resources
    5. Technology Architecture Section: The logical software and hardware capabilities that are required to support the deployment of business, data, and application services. This includes IT infrastructure, middleware, networks, communications, processing, and standards.
    6. Delivery Approach Section: Describes the approach of the solution delivery, what are the phases, would there be POC, would there be a Pilot implementation, who would be doing what, the responsibilities of various internal and external teams.

What do we mean by Architecture?

Architecture is defined as “The fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. An architecture has two meanings depending upon the context:

  1. A formal description of a system, or a detailed plan of the system at a component level to guide its implementation.
  2. The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time

Microservices Interview Questions: 90 Technical Questions with Answers

Just got this Book published on Amazon Kindle check it out at https://www.amazon.ca/dp/B07KMD77YB/ref=sr_1_4?ie=UTF8&qid=1542400692&sr=8-4&keywords=microservices+interview+questions

Wisdom is learning all we can but having the humility to realize that we do not know it all. Microservices Interview Questions 90 Technical questions with clear and concise answers will help you gaining more wisdom in Microservices Interviews. The difference between a great Microservices consultant and someone who kind of knows some stuff is how you answer the interview questions in a way that will show how knowledgeable you are. The 90 questions I have assembled are for: job seekers (junior/senior developers, architects, team/technical leads), and interviewers.

Microservices Interview Questions are grouped into:

  • General Questions
  • Design Patterns Questions
  • API Design Questions
  • Containers and Orchestrations Questions.

Increase your earning potential by learning, applying and succeeding. Learn the fundamentals relating to Microservices based Application architecture in an easy to understand questions and answers approach. It covers 90 realistic interview Questions with answers that will impress your interviewer. A quick reference guide, a refresher and a roadmap covering a wide range of microservices architecture related topics & interview tips.

Sample Questions

  1. Why a Microservices architecture?


Microservices Architecture provides long-term agility. Microservices enable better maintainability in complex, large, and highly-scalable systems by letting you create applications based on many independently deployable services that each have granular and autonomous lifecycles. And Microservices can scale out independently.


Instead of having a single monolithic application that you must scale out as a unit, you can instead scale out specific Microservices. That way, you can scale just the functional area that needs more processing power or network bandwidth to support demand, rather than scaling out other areas of the application that do not need to be scaled. That means cost savings. Microservices approach allows agile changes and rapid iteration of each Microservice. Architecting fine-grained Microservices-based applications enables continuous integration and continuous delivery practices. It also accelerates delivery of new functions into the application. Fine-grained composition of applications also allows you to run and test Microservices in isolation, and to evolve them autonomously while maintaining clear contracts between them. As long as you do not change the interfaces or contracts of a Microservice, you can change the internal implementation of any Microservice or add new functionality without breaking other Microservices that use it.

  1. What is the Eventual Consistency?

Eventual consistency is an approach which allow you to implement data consistency within a Microservices architecture. It focuses on the idea of that the data within your system will be eventually consistent and it doesn’t have to be immediately consistent. For example, in an E-Commerce system, when a customer places an order, do you really need to immediately carry out all the transactions (Stock availability, charging the customer credit card, etc.)? There are certain data updates that can be eventually consistent, in line with the initial transaction that was triggered. This approach is based on the BASE model (Basic Availability, Soft state, and Eventually consistent). Data updates can be more relaxed, and don’t always have to have all the updates apply to the data immediately, slightly stale data to give approximate answers is okay sometimes. BASE model contrasts with the ACID model where all data related to the transaction must be immediately updated as part of the transaction. The system becomes more responsive because certain updates are done in the background and not done as part of the immediate transaction. The eventual consistency approach is highly useful for long running tasks. One thing to note about the eventual consistency approach is depending on the patterns you use, the actual time it takes for the data to become consistent will not be days, minutes or hours, it will potentially be seconds. Eventual data consistency across your Microservices architecture that happens within seconds is acceptable because of the gains you get in terms of performance and responsiveness across your system. Eventual consistency using the right patterns can be so immediate, and preparing for inconsistencies and dealing with race conditions might not actually be such a huge task. The traditional approach to eventual consistency has involved using data replication. Another approach to eventual consistency is Event based which works by raising events as part of transactions and actions in an asynchronous fashion as messages are placed on message brokers and queues.

  1. How to approach REST API Design?

    1. First, focus on the business entities that the web API exposes, what kind of CRUD operations needed. Creating a new entity record can be achieved by sending an HTTP POST request that contains the entity information. The HTTP response indicates whether the record was created successfully or not. When possible, resource URIs should be based on nouns and not verbs like the operations on the resource. A resource does not have to be based on a single physical data item.
    2. Avoid creating APIs that simply mirror the internal structure of a database. The purpose of REST is to model entities and the operations that an application can perform on those entities. A client should not be exposed to the internal implementation.
    3. Entities are often grouped together into collections (orders, customers). A collection is a separate resource from the item within the collection, and should have its own URI.
    4. Sending an HTTP GET request to the collection URI retrieves a list of items in the collection. Each item in the collection also has its own unique URI. An HTTP GET request to the item’s URI returns the details of that item.
    5. Adopt a consistent naming convention in URIs. In general, it helps to use plural nouns for URIs that reference collections. It’s a good practice to organize URIs for collections and items into a hierarchy.
    6. You should provide navigable links to associated resources in the body of the HTTP response message. Avoid requiring resource URIs more complex than collection/item/collection.
    7. Try to keep URIs relatively simple. Once an application has a reference to a resource, it should be possible to use this reference to find items related to that resource.
    8. Try to avoid “chatty” web APIs that expose many small resources. Such an API may require a client application to send multiple requests to find all of the data that it requires. Instead, you might want to denormalize the data and combine related information into bigger resources that can be retrieved with a single request. However, you need to balance this approach against the overhead of fetching data that the client doesn’t need. Retrieving large objects can increase the latency of a request and incur additional bandwidth costs.
    9. Avoid introducing dependencies between the web API and the underlying data sources. For example, if your data is stored in a relational database, the web API doesn’t need to expose each table as a collection of resources.

It might not be possible to map every operation implemented by a web API to a specific resource. You can handle such non-resource scenarios through HTTP requests that invoke a function and return the results as an HTTP response message. For example, a web API that starts some operation such as run validations could provide URIs that expose these operations as pseudo resources and use the query string to specify the parameters required.

Enterprise Architecture and Domain Driven Design

Why do we call engineers who are building Architects? When I visited Rome in May 2018 and visited the colosseum, our guide pointed out that the colosseum arches were the inspiration for the architecture science or the knowledge of how to build arches. How to build arches is a very complex process, it took a lot of effort to rediscover it in the Renaissance times. We call software engineers who design large software architects and those who guide organizations through the business, information, process, and technology changes necessary to execute the organization strategies Enterprise Architects.

Figure 2: Me inside the colosseum

Enterprise Architects define solutions to business problems by creating a solution architecture blueprint. Solution Architecture is the process of defining the design or organization of different systems and components to fulfill a business needs and requirements. By systems and components, I mean software applications, hardware devices the software runs on. With the emergence of Microservices enterprise architects are working on defining the required microservices for building IT assets that enable the organization in achieving it business goals. Domain Driven Design – Bounded Context principle resonates with how to define the actual size of Microservices (“Micro” does not mean tiny in this context). Recently I was chatting with a couple of colleagues about how to define and implement strategies for embracing Microservices and Domain Driven Design (DDD) within a large organization. And the following question came up:

Can we apply Domain Driven Design to existing Mainframe code and legacy Monolithic Applications?

My answer was “Yes we can” just like Obama slogan . DDD is about defining and designing better software that achieves the business goals, has supple design that can be enhanced by adding new features in the future, and can be easily understood by software developers and the business experts. You do not need to adopt new platforms, technologies to do that. You can app the DDD principles to any large complex business solution you are building. Think of DDD as a guiding way to approach a problem by decomposing it into smaller problems and tasks. You can apply that to anything in the universe just listen to Stephen Duneier Ted Talk https://www.youtube.com/watch?v=TQMbvJNRpLE

What is Domain Driven Design?

Domain Driven Design is an approach to combining business analysis with software design to make sure the developed business software meets the business problem it is trying to solve. It is part of the agile software methodologies that focuses on the actual software being built. But unlike extreme programming or the other agile methodologies where the absolute focus is on the code written, Domain driven design focuses on:

  1. Creating a representative model of the software- the focus of this model is on the actual components, classes, packages, modules and relationships that actually solves the business problem at hand rather than the code required to code the software for the chose platform, framework etc.
  2. Making sure that the model is understood by both the Business experts also referred to as Domain Experts and the developers
  3. Making sure the model and the code are in synch

Domain Driven Design introduces several techniques to help with achieving those goals:

  1. Divide the problem domain into several smaller domains called Bounded Context
  2. Bounded Context
  3. Ubiquities Language
  4. Refactoring both the code and the model

To be continued …

Choosing a Platform for a new microservice

Microservices endorses using different technologies to build components of a solution or a system. You can have components based on Java or Scala with Spring or C# & .Net etc. But if you are going to build a new microservice which platform should you use? Most developer’s or architects would choose based on their background if they came from Java they choose Java or .net if they are from .net. This kind of reminds me of an old Dilbert comic in it Dilbert tells the marketing manager that he can draw a fish on the screen. It sounds to me like most architects choose a platform because they are familiar with not because of its merits. I am working on devising a decision tree to select which platform is more suitable for a microservices. So far if you are building a microservice to interact with Hadoop/Spark or Kafka Java/Scala/Spring is the platform to go, though there are lots of libraries that makes it easy to develop such microservices in .Net. I am still working on this but thought I should share it. Let me know your thoughts and why would you choose a platform over the other?

Java Scala /Spring MVC

  • Pros:

    • Respected in the business
    • Widespread
    • Constrained coding style might lead to easier to read code
    • Better ecosystem for libraries and tools
  • Cons:
    • Verbose, both with the language and framework
    • Does not have as good support for threading and asynchrony as C# has with async/await

C#/ASP.NET Core

  • Pros:
    • Asynchronous -oriented
    • Better generics, and integration with LINQ
    • Better IDE integration, and configuration setup
    • Quicker to get things up and running in
    • Able to be component oriented as well as MVC
    • Availability of other programming styles than OOP, including functional.
    • More low-level constructs allowing optimization
  • Cons:
    • Newer, slightly buggy
    • Not as battle-tested as Java
    • Heavy IDE (although you can use Visual Studio Code, too)

Microservices, TOGAF, Solution Blueprint and API Design


While providing microservices architecture consulting services, I found it necessary to extract the Interface Design from the Application Architecture. The Interface Design is usually part of the application architecture. However, the Interface design both UI and API are of extreme importance and value to the solution and to the organization offerings, it requires having a sperate section of its own. With the emergence of Microservices and many companies offering their services through APIs to allow clients to hock in and utilize their services. Or as business/marketing people refer to “Monetizing the API”. I think it is paramount to get the interface design more attention and specifically the API design. That is why I defined this new Interface Architecture phase in my customized TOGAF offering to the organization.

Architecture Type

Description

Business Architecture

The business strategy, governance, organization, and key business processes.

Interface Architecture

A blueprint for the individual application interfaces both User Interface and Application Programming Interface and their relationships to the core business processes of the organization.

Application Architecture

A blueprint for the individual application components to be deployed, their interactions, and their relationships to the core business processes of the organization.

Data Architecture

The structure of the application logical and physical data assets and data management resources.

Technology Architecture

The logical software and hardware capabilities that are required to support the deployment of business, data, and application services. This includes IT infrastructure, middleware, networks, communications, processing, and standards.

 

I would like to know your thoughts on this, and how did you handle the API design in the Solution Architecture Blueprint.

SharePoint Content Management

I met with a previous colleague a few days ago who had move to an IT Research and advisory company. He was working on writing a best-practices recommendation for using SharePoint and the utilization of SharePoint Content Management capabilities in the enterprise data governance. Since, I had worked in many organizations before and utilized SharePoint in many solutions he wanted to get some insight from me. While it is flattering to be the man of insight I had concern about “Best Practices” concept. We are not talking about SharePoint in general which you can get that from Microsoft after all they are the vendor and know it best. In general, I would say there would be best practices would have to be divided into industry sectors (Retail, Banking, Finance, Manufacturing, Technology etc.) and for each organization there will be best practices in the organization context. Maybe that someone is seeking my insight had made me too philosophical so below are the questions he had for me and the answers I gave. Please share how you would have responded to them.

What were the key challenges that SharePoint helped you solve?

I have worked on so many SharePoint projects with many different challenges, A few of the major projects are:

  • City of Vaughn Online Portal implementation which was base on 2007 version. It was used for internal and external collaboration, cooperation and communications with the community of the city of Vaughan residents.
  • Adecco USA which was 2010 SP implementation. This was exposing different portals for different Adecco Brands of Head Hunting web sites, users would search for jobs, post resumes, create profiles and communicate with the Head Hunters through it.
  • For the Government of Ontario, I worked on multiple SharePoint projects building portals for cooperation and processing.
  • For TPS, I have built several SharePoint based applications for automating workflows. The most famous one is the Paid Duty Management system (basically overtime for uniform officers) where officers can apply for Paid Duty. And the system based on the business rules would apply

So mostly SharePoint was used as a platform to build applications on. Allowing me to concentrate on the core for the required Solution.

What architecture did you decide to adopt for SharePoint in your organization (cloud, premise, hybrid)?

Mostly had been on premise, lately with SharePoint 365 is the adoption of the cloud. Any new project is basically being built on the cloud or migrating the on-premise solutions to the cloud

If a migration took place, why did you choose to do this? If greenfield why this architecture?

The migration decision was mostly to utilize Azure.

Did you install and configure any third party add-ins?

On-premises there was many third party add-ins installed freely. On Azure you have to choose from Marketplace. But in general yes, utilizing some of the COTS helps to reduce the complexity of your solution.

Do you use SharePoint for Web Content Management? If so what capabilities are usefull for you?

Yes, basically it was used for providing governance and control over the data and document being added to SharePoint.

What would you say the gaps are in SharePoints Offering for ECM?

There is always gaps in any offering, I can not name one right now.

How did the Web content management functionality work out for you in SharePoint?

It worked ok from the most part.

If you migrated what version did you migrate to and from?

Yes I did several migrations form 2003-2007 which was the hardest as the structure was completely different, basically used third-party tools to facilitate the movement and reformatting the data.

What benefits did you gain from the upgrade?

The newer versions always provides better performance new features etc. so utilizing the advantages of the new version over the older version.

What Backup strategies did you put in place for SharePoint?

This question reminds of an application I have built that would back up SharePoint document to AWS S3. I believe I have built for SharePoint 2007. Below is the flyer I had for this

Enterprise Architecture and Domain Driven Design – Build Custom Business Logic as Microservice or part of the COTS Application?

As enterprise Architects, depending on the organization we are adopting readymade software applications (or COTS) and tailoring or customization of the software to the needs of the organization. The tailoring can involve UI changes, workflows changes, integrations with other systems in the organization and sometimes business logic added or modified. The UI changes usually must happen on the COTS (Component of the Shelf) application and so are workflows unless you are building a UI façade for this application. I saw one organization did build a completely different Web App to provide its employees access to work shift schedules as the Time and Attendance system could not handle the access by the hundreds of thousands of employees and save on licensing and infrastructure costs. This leaves us with Integrations and business logic. In this installment. I will address Business Logic.

Should we build the custom Business Logic as Microservice or part of the COTS Application?

Aha, this is a tough decision. I prefer to put all customizations out of the software package somewhere else. Would it be in Microservices or built as part of the ESB. I want to have the freedom to have the business logic as platform independent as possible. By moving the custom business logic out in webservices implemented by Microservices or part of the ESB I accomplish that. There are political aspects to this issue too? First the COTS vendor wants to have all the business logic customization in its application for the following reasons:

  1. The vendor wants to increase the $$$ from the custom development services they will provide.
  2. The vendor can enrich its application offerings and business logic by taking this customized business logic and generalizing it in future releasee.
  3. The more customizations and business logic embedded in the vendor logic the harder for the organization to switch to another competitor as they will the cost of customizations from scratch would add up.

The delivery department would prefer not to take ownership of new software components, especially if the organization does not have many software developers and depends on consulting companies.

My argument is that, all software be it ERP system like Dynamics AX or Time and Attendance systems is generic and the vendor offer it to your organization and your competitors. What differentiates your organization from your competitors is the custom process and business logic your organization uses. If you let the vendor build these customizations in their software then you end up subsidizing your competitor upgrade, and maybe giving them your edge. That is why I tend to recommend extracting any custom business logic new or modified and building it outside the COTS. Sometimes this is not possible due to the CTOS platform limitations or due to organizational standards. But whenever is possible extract any custom business logic outside of the COTS

Let me know your thoughts on that.

Enterprise Architecture and Domain Driven Design – 1

Why do we call engineers who deign building Architects? When I visited Rome in May 2018 and visited the colosseum, our guide pointed out that the colosseum arches were the inspiration for the architecture science or the knowledge of how to build arches. How to build arches is a very complex process, it took a lot of effort to rediscover it in the Renaissance times. We call software engineers who design large software architects and those who guide organizations through the business, information, process, and technology changes necessary to execute the organization strategies Enterprise Architects.

Figure 2: Me inside the colosseum

Enterprise Architects define solutions to business problems by creating a solution architecture blueprint. Solution Architecture is the process of defining the design or organization of different systems and components to fulfill a business needs and requirements. By systems and components, I mean software applications, hardware devices the software runs on. With the emergence of Microservices enterprise architects are working on defining the required microservices for building IT assets that enable the organization in achieving it business goals. Domain Driven Design – Bounded Context principle resonates with how to define the actual size of Microservices (“Micro” does not mean tiny in this context). Recently I was chatting with a couple of colleagues about how to define and implement strategies for embracing Microservices and Domain Driven Design (DDD) within a large organization. And the following question came up:

Can we apply Domain Driven Design to existing Mainframe code and legacy Monolithic Applications?

My answer was “Yes we can” just like Obama slogan . DDD is about defining and designing better software that achieves the business goals, has supple design that can be enhanced by adding new features in the future, and can be easily understood by software developers and the business experts. You do not need to adopt new platforms, technologies to do that. You can app the DDD principles to any large complex business solution you are building. Think of DDD as a guiding way to approach a problem by decomposing it into smaller problems and tasks. You can apply that to anything in the universe just listen to Stephen Duneier Ted Talk https://www.youtube.com/watch?v=TQMbvJNRpLE

What is Domain Driven Design?

Domain Driven Design is an approach to combining business analysis with software design to make sure the developed business software meets the business problem it is trying to solve. It is part of the agile software methodologies that focuses on the actual software being built. But unlike extreme programming or the other agile methodologies where the absolute focus is on the code written, Domain driven design focuses on:

  1. Creating a representative model of the software- the focus of this model is on the actual components, classes, packages, modules and relationships that actually solves the business problem at hand rather than the code required to code the software for the chose platform, framework etc.
  2. Making sure that the model is understood by both the Business experts also referred to as Domain Experts and the developers
  3. Making sure the model and the code are in synch

Domain Driven Design introduces several techniques to help with achieving those goals:

  1. Divide the problem domain into several smaller domains called Bounded Context
  2. Bounded Context
  3. Ubiquities Language
  4. Refactoring both the code and the model

To be continued …