Category Archives: SharePoint

Decision time – SharePoint – On-Premises, Hosted or Office 365?

In the middle of 2011, Microsoft introduced Office 365 – the new way how organizations will use MS Office suite  in the future, at least how Microsoft hopes. The competition is getting quite stiff in the office productivity tools based on the cloud solutions, and the new Microsoft’s flag product is key to its survival. Office 365, depending on the type of selected license, includes MS Office Plus suite, SharePoint, Outlook and Lync (instant communication solution). With significantly reduced need for maintenance, infrastructure and security, one would expect that this is an ideal solution in particular for small and medium organizations.

Not so, according to Richard Harbridge. Richard did an excellent job in painstakingly collecting data and developing model comparing the costs specific for implementation of SharePoint portion of the suite, evaluating on-premises, hosted and Office 365 installations. The results of the model suggest that the user volume/licensing heavily influences Office 365 competitiveness, and in some cases Office 365 is outright more expensive than on-premises installations, even though implementation and maintenance costs are lower. Office 365 business model clearly targets small business with smaller number of users.

The research looked at several costs, spread over 1 to 5 years:

  • Professional services
  • Infrastructure or Hosting
  • Administration team
  • Licensing

For comparison there were several typical configurations taken into account:

  • Single server (web server and MS SQL server on virtual machine)
  • Small farm standard (two servers: separate web and MS SQL server installations)
  • Small farm with high availability (2 web servers and 2 MS SQL servers)
  • Medium farm (1 web server, 1 application/indexing server, 1 MS SQL server)
  • Medium farm with high availability (2 web servers, 2 application/indexing servers, 2 SQL servers)

As you can see below, number of users (that translates into licensing costs) plays significant role in costs structure, favoring organizations with 100 or less employees. This makes me thinking that Microsoft purposely is trying to keep medium and larger organizations away, due to limitations of their current infrastructure and support. If so Microsoft will soon will have to address it, as Google with their Google Docs is well on the way to take over the market. Overall – the cloud version of the productivity software for organizations seems to be the right direction, and we should see more and more vendors getting into this space.

To see Richard’s research, visit his blog:


SharePoint and Information Security

Interesting survey was recently published by Cryptozone on SharePoint security. The results are evidence of need and importance of information management governance and proper, upfront design of the information systems. It appears that in most of organizations, the responsibility for assigning of the access rights to SharePoint documents still belongs to IT Administrators, as it was indicated by 69% of respondents. At least this segment of users knew who was in charge; in contrast to 22% who did not even know who managed it. The problem with ceding of the responsibility for content protection entirely to IT is that IT primary focus is on maintenance and configuration of the technical infrastructure, but with limited knowledge and understanding of the content and its specific protection needs. IT cannot and should not make decisions on how particular type of information should be protected, and who should have access to it.

So who should be responsible for making such decisions? The answer seems to be intuitive – the business – but 43% of respondents said that they do not trust document authors to control who should read their documents. This would indicate that most of the users have low levels of awareness and understanding of the security needs. This seems to be confirmed by another set of responses that indicated that over 45% of users did copy sensitive and confidential information to unprotected USB memory sticks and emails. 55% of these respondents claimed that reason for this was the need for sending necessary information to users without access to SharePoint, with further 43% needing it for working at home. Over 30% of users were more concerned about getting the work done rather than security, and another 47% did not even think about security or did not care.

One of the contributing factors leading to taking documents out of SharePoint’s control, was the need to share it with third parties – over 56% of respondents said that their organizations did not have external portals to help with collaboration outside of the organization.

The bottom line is that this exposes the organizations to risks including legal risks and intellectual property theft. Therefore proper solution would be to give some thought before SharePoint is rolled out, answering questions on how the information is going to flow across the organization, how it is going to be accessed, how users will be segmented by their needs and how it is going to be protected. This should lead to development of information management governance, that would clearly describe roles and responsibilities across the organization, and ways how the information should be distributed and protected. Lastly, the most important step is to make the users aware of the security needs, training them on the policies and periodically reinforcing this knowledge.

Business Process Management key to successful implementation of information management

Business processes are integral part of information management. In organizational context they could be compared to cardio-vascular systems in living organisms, with blood being represented by information, and the processes by structure of veins and valves. Like with the organism, inefficient circulation will lead the organization to poor performance, inability to compete, which as end result could be fatal. Business processes could be defined as a set of related, structured activities and discrete tasks, moving and enhancing business information to achieve specific goals and objectives. They could be divided into three groups:

  • Management processes – governing operations of the organization often called ‘corporate governance’
  • Operational processes – set of core business activities to generate value and revenues, like manufacturing, purchasing, sales, or marketing
  • Supporting processes – set of auxiliary activities supporting the core, operational processes, for example HR, accounting, information technology, and support

The processes exhibit certain common characteristics:

  • Definition – they have clearly defined scope,  inputs and outputs
  • Sequencing – the activities could be sequenced and prioritized for execution
  • Benefactor – there must be specified recipient of the process outcome
  • Value – adding value during the process of transforming or carrying the data
  • Inclusion – they exist in the context of the organization
  • Cross-functionality – the process often spans multiple functions within organization

There are two concepts related to process management: Business Process Management (BPM) and Business Process Reengineering (BPR). Although both deal with the process control and flow of information, and sharing many common characteristics, there is however a significant difference between the two. Business Process Management is an ongoing initiative with set of operational activities to capture, define, monitor, and to gradual improve the organizational benefits. BPM is often implemented using bottom-up approach, and it introduces more gentle change to the organization. Business Process Reengineering on the other hand, is more project oriented, with clearly defined end-state and timeline, redesigning the processes and transforming the organization. It is often implemented as top-down approach and requires much stronger organizational change management activities on many fronts within the organization. BPR initiatives could create lot of apprehension among the workers, due to introduced change in work habits, and often their key success measurement relates to reduction of the workforce.

Formalization, standardization and automation of business processes can introduce several benefits to the organizations:

  • Better utilization of organization’s workforce
  • Improved process speed
  • Reduction in number of errors
  • Costs reduction
  • Risks reduction
  • Improved customer service
  • Duplicate work reduction
  • Improved visibility of the processes and their efficiencies

Formal business processes implementation might need to resolve several issues:

–   Staff resistance to change – new processes will impact the ways how the work is done right now, and could introduce fears related to exposing potential inefficiencies, resulting in workforce reduction, or transferring to other departments

  • Implementation time is often lengthy due to need of discovery and documentation of hidden, informal processes, and their adjustments
  •  ‘Butterfly effect’ – any small inaccuracies in identification of the sub-processes, could translate into larger problems down the value chain
  • Difficulty in finding skilled resources to deliver
  • Insufficient funding – most of organizations face budgetary constraints today, while the business process changes often require substantial time and money commitments
  • Lack of management support – formalization and automation of business processes might not be on the top of management’s priority list.

The process automation could be categorized by complexity of the implementation, and organizations could select one or more depending on their needs:

1. Routing

Routing is the simplest implementation of the business processes, addressing ad-hoc needs of the end users. Usually they linearly move information from person to person, without integrating with information generating or consuming applications. They are often employed for user notification about waiting task and monitoring of the completion status. The users need to open and process the tasks manually. That type of solution gives limited ability, if any, to implement rules associated with the process.

2. Workflow

Workflow is more sophisticated implementation of the business process automation. Among others, it allows running processes not only serially but also in parallel, saving time and improving productivity. The processes can also have defined complex set of rules, exceptions and conditions. Often there is a graphical user interface that allows for easy customization of the workflow. Useful feature of workflows is the ‘role’ concept allowing assigning tasks to roles rather than to specific people. In cases where user is unavailable, a rule could assign the task to another person, belonging to the same role. Completion of the task could trigger next step in the process chain.

3. Business Process Management

The Business Process Management is extending this concept further, to the whole enterprise, allowing crossing platforms, applications and repositories. It addresses complexities of the cross-departmental processes, and allows for their standardization. Implementation of automation requires identification of core practices and detailed analysis of business rules and triggers. Flowcharting and process modeling are two of the techniques used for this purpose. Flowcharts are graphical representation of sequence of steps and decision branches. They are excellent tools to provide blueprint for implementation, as well as could serve as communication and change management instruments. Process models on the other hand, are more elaborate tools adding intelligence, dependencies and levels to the process tasks. The simulation functionality allows identifying and resolving potential bottlenecks, inefficiencies and loops. Integration and operational monitoring of the processes could help with continuous improvement. Since the implementation of BPM is much more complex than with other two categories, it requires careful planning, change management and funding.

Master Data Management and Governance

DataMicrosoft SharePoint 2007 and then 2010 triggered rapid rates of adoption of collaboration and document management systems. Soon many organizations painfully realized the importance of Information Governance. Without it, the implementations quickly became digital landfill, just replacing but not improving shared drives problems. Often departments started building their own sites, with their own branding, cumbersome and unmanageable security structures, own metadata, poor or entirely missing taxonomies, leading to state of mess where users couldn’t find anything. Even worse, duplication of documents led to confusion, the business decisions based on outdated data, the storage size and backup costs exponential increase, and deterioration of systems performance. Worst of worst, since information was not purged or when it was, it happened randomly, this exposed the organizations to e-Discovery related legal risks and litigation costs.

To address these problems, organizations needed to develop set of aligned governance constructs within an overall Information Governance Framework. Among those constructs are Information Security Governance, Information Architecture, Data Quality, Records and Retention, Master and Reference Data just to mention few. I think that the latter plays very significant role and should be done early to get information under control.

So how Master Data Management could be defined? It is a set of processes, tools and organizational structures, where business and IT work together to address issues likes uniformity, accuracy, stewardship, and consistency and accountability of the organization’s data. This leads the data to become authoritative, secure, reliable and sustainable.  But not all data should get the same level of attention.  Master data is a ‘key’ data gathered and used by multiple departments during operations of the business like for example – customer data, information about products, employees, materials and so on. Master Data must contain most accurate and authoritative data available, and serve as single source of truth across the organization. Lot of organizations however find it difficult to secure the necessary funding and support from senior management, due to difficulty with measurement of return on investment.

Earlier this year, Gartner published some predictions related to Master Data Management governance and impact on organizations by end of 2016:

–          Only 33% of organizations that initiated MDM will be able to demonstrate its value. The difficulty here is that such initiative must present complete approach and be an ongoing process rather than once-off isolated project. This means that there needs to be consensus among senior executives and obtaining this is often quite challenging.

–          Spending on information governance must increase fivefold to be successful – and as per point above, needs to include other disciplines within the Information Management Governance Framework like quality management, lifecycle and retention, privacy and security. This will lead to building larger teams focusing on the governance and higher costs.

–          20% of CIOs in regulated industries will lose their jobs failing to implement information governance. IM governance is a construct that allows organization for compliance with regulations, and the primary responsibility for this lies with CIO and Legal Counsel.  Breaches in information security, leaks of confidential information, and breaches in privacy will lead to reputational and financial damage to those organizations.

The good news is that lot of organizations already recognize these risks, as according to Gartner, last year they have seen 21% increase in spending on MDM.


Office 365 offers entry point to the Cloud but with limitations

Office 365 is making steady progress in capturing small and medium business market segments with its software-as-a-service office suite and especially with cloud based version of SharePoint 2010. Adoption in larger enterprises is much slower however. For lot of organizations Office 365 is an excellent entry point into Cloud services that allows reduction of operational costs, physical storage requirements, and more optimal use of support resources. This all translates into reduction of total cost of ownership, in addition to elimination of more intangible headaches and risks like software updates or upgrades. However, quite a few organizations still have concerns related to security, reliability, ownership of data, privacy, or lack of knowledge what to do with existing on-site installations and investments. Honestly speaking, with regards to security or reliability – for most of organizations, cloud services are usually better in those areas than in-house operations. Cloud companies like Amazon, Microsoft or Rackspace have whole teams dedicated to these subjects, monitoring servers 24/7. Regarding ownership of the data, this shouldn’t be an issue either, since the data is not shared, even in multitenant environment (Microsoft offers two models – multitenant and dedicated, the latter might be an option for those who are obsessed with information protection). Deciding what to do with existing SharePoint installations, and the privacy – are valid concerns. In some countries (Canada is one of them, and so is European Union), passing information that includes personal data of users or clients across country borders, is illegal. Recently Microsoft announced cloud solution that would secure and limit the boundaries of the information transfer specifically to address government requirements, but so far this is limited only to the US. Also, the Microsoft SharePoint offering that is part of Office 365 suite, does not provide all the features that on-site installations have. Some of them:

  • Lack of FAST search solution
  • Lack of integration with Microsoft Information Rights Management
  • Lack of ability to index external databases from SharePoint search
  • Lack of Performance Point Services
  • Lack of support for external lists

So, for organizations that need more sophisticated configurations, this might not be the best option – at least for now.

But there is however another possibility – companies that really want to move into cloud, could try hybrid solutions. Assuming that such organizations have good information architecture and defined business processes, they could partition data and processes in such way that critical information is handled by in-house installations, and the rest is stored and processed using cloud solution. The integration of the data might require building a mash-up portals for the end users, so it would require some good thinking before implementation, and solid governance in place. It is important however to understand limitations of such solution – for example – federated search based on cloud and on-premises data will not work. Key success factor for such implementation would be a solid understanding of the business requirements, and alignment with overall long term goals of the organization. There are however quite a few benefits that cloud solutions bring and Microsoft is working on closing some of the gaps.

Implementation of Records Management in SharePoint 2010 is not trivial

DecisionRecords management implementation in SharePoint is not a trivial thing. I wrote about this on couple of occasions in the past. Earlier this week there was an interesting presentation from ARMA, expanding on some of these topics.

First of all – SharePoint out-of-the-box implementation will provide only a partial and rather informal – records solution. Many people consider Department of Defense DoD 5015.2 records management requirements as an overkill. This might be true for most of non-governmental organizations, although ARMA identified that of 168 requirements in DoD 5015.2, at minimum 105 are considered as those that make system a robust records management application. SharePoint 2010 satisfies 72 of these requirements. That leaves gap of 33 requirements that needs to be addressed. There are two ways of doing this – getting SharePoint implementation customized or getting a third party add-ons to handle the records management. Both of the solutions have their own pros and cons related to costs, licensing, training and operational support requirements.

Among the issues that need to be addressed are:

–          Centralized file plan, linked to a retention schedule. I wrote about this earlier – this requires usage of records center rather than in-place records management.

–          Securing, management and maintenance of the file plan by the records managers. This includes securing top levels of the file plan hierarchy but with ability to allow delegated departmental records clerks to create and maintain third level of subject and case file folders.

–          Proper disposition process – SharePoint OOTB handles automatic deletions, but disposition process needs to be customized, including records qualification, reviews, approvals, cutoff times, and records state status updates

–          Distinction between the subject records and the case file records. The significant difference between the two is related to the above process, where the entire content of the Document Set in case file record must be disposed at the same time, preventing the users from destroying the record partially.

–          Centralized management of Information Management Policies in SharePoint, due to required security levels. Information Management Gallery is not enough, and this also impacts ability to implement in-place records management, where control of these policies and maintenance of the security becomes quickly impractical.

–          Ability to monitor ingestion of records, their classification status, and retention events. This includes bulk uploads and changes to records metadata. Even on document level it is currently a huge pain in SharePoint.

–          To manage the records across their lifecycle, proper metadata must be collected and updated along their way. The specific records related metadata needs to be defined and implemented during the rollout.

–          MS Outlook integration with ability to declare emails with their attachments as records, and ability to add records specific metadata.

In either case – customization of SharePoint or integration of third party add-ons requires lot of thought planning, and tough decisions making.

SharePoint – Records Center or In-Place Records Management?

Folder - records managementSharePoint 2010 brought some new capabilities but at the same time challenged the implementation teams with making some tough decisions. One of them is – how to implement records management. In MOSS 2007 – it was simple; the only possibility to achieve the functionality was through setting up Records Center site. In this case, for the content to be declared as a record, it had to be moved to separate storage area. SharePoint 2010 now offers In-Place Records Management – content that was declared as the record stays where it was originally, but the additional information management policies need to be applied to make sure it is immutable. Which solution is better? Which one should be chosen?

As expected there is no simple answer to this question – it depends. But once the decision is made, the organization needs to live with its consequences. The way back is costly and time consuming, it makes reversing the course usually unfeasible. So what are the pros and cons of either solution? The list below captures some of the key differences and their potential impact. Please note that some of the functionality was split to reflect the fact that business users and records managers are often driven by conflicting requirements – ease of filing, access, finding information and ability to collaborate for business users and ability to restrict access, protection and enforcing retention rules for records managers.

Feature In-place Records Center Comment
Retention Implemented through information management policies by content type. It might provide more flexibility in getting the rules more granular but at the cost of maintenance complexity. Simple – once record is placed in its bucket, it inherits its retention rules. Most of business users are not concerned by the retention; this is of primary interest to records managers. However what needs to be taken into account, if implementing in-place records management, the records lifespan might be longer than the hosting site. This creates potential problems with records preservation when the site needs to be disposed. This could lead to tendency to keep obsolete sites live, exposing the organization to legal and regulatory risks, and increased storage costs.
Security/Accessibility No ability to restrict access to records, the record maintains the same visibility across its lifecycle The content visibility and the ability to see its existence in search results can be restricted This could be a concern for records of sensitive nature especially in areas of HR, and Legal departments, or in case of mergers and acquisitions.
Findability of information – business user perspective Excellent, since records reside within their context in their corresponding libraries and folders Might be poor, since same content types reside in the same buckets. This category addresses primarily needs of business users – to locate quickly and easily the information. Since in case of in-place implementation, records are preserved at their source, it is easy to locate the information through its context. In case of the Records Center implementation, the key success factors are related to good governance policies, their implementation, as well as rich and good quality metadata.
Findability of records / eDiscovery – records manager perspective Usually good, though the search needs to span multiple sites Good since all records are located in Records Center, but eDiscovery will require search in both sites and in Records Center In case of Records Center good quality of metadata is important. eDiscovery of records in Records Center is fairly straightforward and quick, however since eDiscovery covers any content – declared as records or non-declared, it will not eliminate need of searching across all locations.
Ease of records management Complex since records are spread across various sites, libraries and folders Easy since records reside in central location with common sets of rules Managing records declared in-place might become messy. Strict governance and control of granularity of information management policies is required. The governance must include cases how to handle records if their survivability exceeds the site lifespan, as well as defining of who can un-declare or supersede records per site. Auditing of the records management and records reporting becomes more complex.
Ease of site management Complex – since sites contain both mutable and immutable content Simple – sites contain only documents that are not yet declared as records, or stubs to Records Center content Sites with in-place records management become more difficult to manage due to differences in how records and transitory documents are handled. Strict governance is required.
Ability to audit records More complex Simple Ability to audit records in in-place implementation depends on each sites audit policies implementation. There are no out of the box compliance reports available. Strict governance is required.
Administrative security By site administrators By records managers In in-place implementation, site administrators have ability to manage both transitory documents and records. This might not be desirable in case of organization in heavily regulated industries, where single responsibility for preservation of records resides with records managers.
Storage Transitory documents and records reside on the same storage medium Scalability could be easily ensured by placing records on separate storage medium In-place implementation might lead to increased storage requirements for both documents that are being actively collaborated and records that might be rarely accessed. Performance issues, security and organizational disaster recovery requirements must be taken into account (this is not the same as simple backups).
Declaring of Document Sets as records Yes No Current version of SharePoint does not allow for declaring Document Sets as records in Records Center


So how to determine which one is more suitable for given organization? There are several factors that will ultimately influence the decision, like:

–          Company culture – strict or more relaxed

–          How heavily regulated is the industry

–          What are the legal, regulatory and statutory requirements

–          Existing processes for handling records – is there already dedicated staff to manage records?

–          Business continuity planning requirements

–          Existing business processes – are document sets best suitable in the organization (this is weak point however, as I am sure that Microsoft is going to come with solution for Document Sets handling soon)

–          Information growth rate and proliferation of sites and sites collections

Decision on the method of records management implementation should not be taken lightly as it will have long term impacts on costs, change management, user adoption, governance, sites and records management, compliance and others. There is no easy way back.