Tuesday, October 18, 2011

SOA Service Cost Estimation Modeling and Approach


Introduction

Cost estimation in Service Oriented Architecture (SOA) based software development efforts is more complex than other traditional non-distributed and silo-ed (web, desktop based rich internet applications etc) applications. In SOA projects, services are generally built using well-defined, reusable, and loosely coupled set of components. Services also comprise of logically grouped functionality and they are exposed to the rest of the world via a contract (Web Service WSDL or JMS Message Queues for example).

The loose coupling brings complexity when it comes to cost estimation as the project teams have to understand and establish all the component level relationships and dependencies involved. Because of the resulting uncertainty, traditional models such as COCOMO or Function Point Analysis cannot necessarily be sufficient for estimating costs of building SOA services. Cost estimation for developing building blocks of services must be done at the component level in order to truly reflect the activities of development involved. For combined components or choreographed services costs must be re-composed in a hierarchical form.

In this study, we propose estimation of costs for SOA services in a recursive fashion. For each component, a detailed work breakdown structure (WBS) that includes analysis, design, development, testing, deployment, integration, support and maintenance activities will be developed. Finally, we propose to include runtime and service level agreement (SLA) management costs if hosting is involved. Further, based on the timeline involved uncertainty in the requirements may be induced because of scope creep once the costs are calculated. For SOA services with longer requirement gathering periods and in whose cases where requirements are subject to change, we propose an additional factor increase in the time and cost of developing the project. However, this factor increase shall be determined directly by the operational managers that are experienced in running the operations within the organization. 

Estimation and Modeling Approach

Classification of SOA services is vital to any SOA governance at organization level as this helps to establish the types of services that the management is interested in investing. Among other types, few examples include data services, infrastructure services, and business services. Other technical categories include web services, file transfer services, identity and access services. Once the categories are finalized they are also governed (managed for changes or version controlled) in a corporate registry. While these types are applicable for implementation and development, we propose to use a different classification model for developing cost estimates. These categories include:
  1. New Services – A service that will be developed from scratch.
  2. Migrated Services – A service that already exists in the organization or its partners, but needs to be elevated to a different standard or technology stack.
  3. Integrated or Choreographed Service – A service that is integrated with yet another service in order to implement larger set of functionality without having to develop from scratch.

Service development may involve one or more aforementioned categories. Project teams (architects and business analysts) must undertake a discovery effort and determine relevant categories before developing a WBS for estimating costs. Once the determination is made, an effort must be put in place for determining all significant components. Every organization adopts a methodology for deriving a list of components. These methodologies could be used for this. We used Service Component Architecture (Apache Tuscany, IBM and Oracle SCA for various projects) reference modeling very effectively for this. A high level approach for achieving this is described in the graphic below:

Figure 1: SOA Service Breakdown Structure for Cost Estimation

Once a detailed list of components (BPEL Human Tasks, Web Services, JMS Queues, XSL Transformations, Security Access Control Lists etc) is developed we propose that project teams develop a high level or detailed (depending on the phase of the project) WBS as described below. The costs are primarily divided into two cost categories:
  1. SDLC costs for software implementation
  2. Hardware runtime costs (In this example a 5 year time period is used for deriving runtime operating costs and this needs to be adjusted by senior executive stakeholders on the project)

Tasks listed below can be categorized into the following:
  1. service discovery tasks – break down of components
  2. component design and architecture tasks
  3. component development tasks
  4. component migration tasks
  5. service integration tasks
  6. Other SDLC activities
 Figure 2: Cost Estimation Spreadsheet

Significant terms in the WBS can be elaborated as detailed below:
  • Component – List of components and other Software Development Lifecycle (SDLC) items that are identified through modeling efforts
  • High level Categorization for cost components – Software for capturing SDLC costs and Hardware for operational runtime costs
  • Tasks – Detailed listing of tasks that are required to develop each component
  • LOE – Level of effort either simply estimated by subject matter experts or from models such as the PERT
  • Risk – A factor that must be derived by the architect or the project manager based on knowns and unknowns.
  • Total LOE – Adjusted LOE per risk factors
  • Cost per Hour – Determined by senior executive managed of responsible organization
  • CPU Hours – Determined by the architect and software vendor for a given hardware, estimated payloads and SLAs
  • Costs per CPU Hour – Determined by the hosting provider based on their infrastructure management costs
  • Total Cost – Determined primarily by (Total LOE * Cost per Hour) for software SDLC and (CPU Hours * Costs per CPU Hour)
  • Overall Total Cost – Sum of all costs involved

Conclusion and Extended Analysis

Cost estimation of SOA services is complex because of the nature of loose coupling and complexity involved. We developed an approach to address the complexity by breaking down the list of activities involved that will incur significant costs. This breakdown approach should be inline with SOA itself without which significant deviation of costs may occur in estimation. Service component architecture and modeling is a prominent approach that is used by SOA practitioners for deriving components in a SOA service. In this study we propose to use this approach, however this can be supplemented and tailored using other approaches if any are adopted by the organization. In particular, tailoring should be applied for incorporating specific risks at the organization or group level if any. In addition to the methodology and development of work breakdown structure described above, we propose that a schedule with timeline and resources loaded be developed. This effort will produce a more reliable cost overall as it will include the resource management (matrix or vertical) aspects into the overall cost model.



Wednesday, October 5, 2011

OAM OIM 11.1.1.3 Intranet/Extranet User Separation

Every large organization that has extranet presence and that employed Oracle Identity Management 11.1.1.3 would like to organize their LDAP directory structures differently. This post provides an overview of how this can be achieved. For more details, please contact us at info@thekpsoft.com. The configuration varies based on a cluster or single server infrastructure, and how relevant access managers are laid out. This post focuses primarily on the underlying OID/OVD configuration and their linkages to OAM/OIM. what we have is two oams, two oims, single oid all replicated in a weblogic cluster (two each).

1. Create another OVD using opmnctl createcomponent -componentName ovd3 -componentType OVD -admin cn=orcladmin -isAdminSSL true -ovdAdminPort 8900 -namespace dc=acme,dc=com -ldapPort 2389 -ldapSport 2636

2. Create another OVD using opmnctl createcomponent -componentName ovd4 -componentType OVD -admin cn=orcladmin -isAdminSSL true -ovdAdminPort 8901 -namespace dc=acme,dc=com -ldapPort 3389 -ldapSport 3636

3. Create another OVD using opmnctl createcomponent -componentName ovd5 -componentType OVD -admin cn=orcladmin -isAdminSSL true -ovdAdminPort 8900 -namespace dc=acme,dc=com -ldapPort 2389 -ldapSport 2636

4. Create another OVD using opmnctl createcomponent -componentName ovd6 -componentType OVD -admin cn=orcladmin -isAdminSSL true -ovdAdminPort 8900 -namespace dc=acme,dc=com -ldapPort 3389 -ldapSport 2636
4. Create cn=external, cn=users,dc=acme,dc=com
5. Create cn=internal, cn=users,dc=acme,dc=com
6. Create cn=system, cn=users,dc=acme,dc=com
7. Using ldif scripts create oamadmin2, xelsysadm2, and weblogic2 in system container, Add respective roles in OID and OIM.

8. Create the following Adapters in the OVD3 with
a. User Adapter with remotebase and root as cn=external, cn=users,dc=acme,dc=com
b. User Adapter with remotebase and root as cn=system, cn=users,dc=acme,dc=com
c. User Adapter with remotebase and root as cn=Groups,dc=acme,dc=com
d. For each user adapter plugin set oamenableflag=true

9.  Create the following Adapters in the OVD4 with
a. User Adapter with remotebase and root as cn=internal, cn=users,dc=acme,dc=com
b. User Adapter with remotebase and root as cn=system, cn=users,dc=acme,dc=com
c. User Adapter with remotebase and root as cn=Groups,dc=acme,dc=com
d. For each user adapter plugin set oamenableflag=true
e. please note that in case internal adapter is linked to AD, difference plug-in configuration will be required.

10. Create the following Adapters in the OVD5 and OVD6 with
a. User Adapter with remotebase and root as cn=users,dc=acme,dc=com
b. Change Log Adapter with remotebase and root as cn=changelog
10. log into weblogic console,
a. navigate to security realms, myrealm, providers
b. add two OVD3 and ovd4 authenticators
c. make sure the order is changed and these should be above the original ovd authenticator

11. Login to each OIM as xelsysadm
a. Click advanced
b. select manage it resource
c. search for Directory Server and change the port number on OIM1 to 2389 and the second one to 3389

12. Login to OAM1 and make ovd3 as primary user store

13. Login to OAM2 and make OVD4 as primary user store

BI Publisher Integration with Oracle OIM 11g

Introduction
Oracle BI Publisher 11.1.1.3 is used to build the reports for ’s OIM environment. Primary schemas that are used for gathering data are Dev_IAU (audit DB) and Dev_OIM.  Audit DB data collection steps will be captured in a separate post. Reports are developed using Oracle BI Publisher plugin for MS Word. Oracle OIM 11.1.1.3 software provides several out-of-the-box reports, these reports are built for Oracle BI Publisher 10.1.3 environment. After running an upgrade utility some reports appear to work, but overall the upgraded reports and the underlying XML documents (xdms, xdos) are corrupt. Reports that are mentioned in this post have been re-created using the same underlying data model and associated queries.




Reports Name
Description
User Reports
 User Authentication Summary
User Login, Logout Information
 User Summary
Key elements of a user account profile (first name, last name etc)
 Users Deleted
User detail that is deleted by date
 Users Disabled
User detail that is disabled by date
 Users Unlocked
User detail that is unlocked by date
Password Reports
 Password Expiration Summary
User account password expiration summary
 Password Reset Summary
User account password resets summary
Role and Organization Reports
 Organization Details
Organization name, creation date
 Role Membership History
Role membership information
 Role Membership Profile
Role membership information
 Role Membership
Role membership information
 User Membership History
Users grouped by role information



 

BI Weblogic Admin Server’s Security Realm Configuration


1.       Take a copy of config.xml in /share/orabi/admin/BIDomain/aserver/bifoundation_domain/config folder

2.       Login using BI weblogic console as weblogic user (or with similar privileges).

3.       Navigate to security realms -> my realm -> Providers tab


4.       Create and order the providers (OAMIdentityAsserter, OVDAuthenticator, DefaultAuthenticator, DefaultIdentityAsserter) as shown in the graphic above. (note: creation of the security providers is out-of-scope for this document).
1.       Login to OIM as xelsysadm user. -> click Administration -> click Create User Icon and create a BISystemUser as shown in the graphic below:
2.       Logout as xelsysadm and login as the user that was just created to reset the first password and answer security challenge questions.
2.       Login to qa-oim1 linux box as orabi, cd to /home/orabi/oracle/middleware/wlserver_10.3/common/bin
3.       Run ‘wlst.sh’ program
4.       Enter the following at the shell prompt - connect ('weblogic', 'iamq$1', 't3://<hostname.domain>:7031')
5.       Run the following commands (replace password below with the most recent password set above):
a.       listCred(map="oracle.bi.system",key="system.user")
b.      deleteCred(map="oracle.bi.system",key="system.user")
c.       createCred(map="oracle.bi.system",key="system.user", user="BISystemUser", password="Welcome_2")

Adding users to BI Consumer Role in Weblogic EM

1.       Login to em at http://<hostname.domain>:7031/em as weblogic user
2.       Navigate to ‘Weblogic Domain’ -> bifoundation_domain
3.       Right mouse click on bifoundation_domain and select security -> Application Roles
4.       Under Search subsection ->  Select application Stripe to Search radio buttong and select obi from the drop-down selection list as shown in the graphic below:


1.       Click Search beside the Role Name and select BIConsumer



1.       Click Add User and search for users for whom you would like give report read access to. For example in the screen above, RT2@abc.com (another OIM user in the MPR repository) is added to the BIConsumer role.

Creation of Data Sources in BI Weblogic Servers


1.       Login to http://<hostname.domain>:9714/xmlpserver/ as weblogic -> click administration as shown below:

1.       Click jdbc -> data sources -> Add Data source and create a data source call oimReportsDS as shown below


1.       Repeat the same and create OIM JDBC data source

Adding Data Sources to BI Consumer Role




1.       Navigate to Administration -> Security Center -> Roles and Permissions


1.       Click BIConsumer -> Add OIM JDBC and oimReportsDS role from available data sources to Allowed data sources and click Apply.

Uploading of Reports


1.       Login to qa-oim1 and FTP all the reports (xdo and xdm folders) to the following directory (/home/orabi/admin/BIDomain/aserver/bifoundation_domain/config/bipublisher/repository/Reports)

2.       The folder location can be found using steps below:

a.       Login as weblogic in http://<hostname.domain>:9714/xmlpserver/
Click Administration -> Server configuration -> check the ‘Path’ value under Catalog


Adding Permissions for Reports and Relevant Data Models


1.       Login as weblogic into http://<hostname.domain>:9714/xmlpserver/

2.       Click Catalog and select report folders in the left menu

3.       Select each folder and do the following for each:

a.       Select more under each report and click permissions


b.       Click + sign to add BIConsumer roleand select read, wirte, Run Report Online, View report online checkboxes and click ok

c.       Perform the same for relevant data model element and select the options as above.
d.       Login as user with BIConsumer roles and you should be able to access reports.