Management in Big Data Projects

Information security should be one of the topmost priorities for a risk management team in a big data project. Since the infrastructure for a big data project is cloud based and is usually provided by a third party vendor as such checking the security contracts and breach resolution is very critical for a seamless project delivery.

To avoid privacy breach with the client’s confidential data the solution providers working in a big data project need to create a Privacy Impact Assessment Program. Such a program ensures privacy by design and has a mechanism of proactive monitoring and pre-prepared contingency plans. This program will ensure that privacy would not come in just as an added feature of a risk management plan but would be evolved as a core functionality delivery. On top of its effectiveness, such a program needs to make certain that privacy policies for the users are transparent and facilitates user-friendly options.

2. Project Manager’s requirement for high technical skill set

A typical project management role for a big data project has a good amount of dependencies on technical aspects- data architecture, technical metrics and KPIs, developing platforms as Hadoop and NoSQL, measuring a project’s maturity to take up a big data based solution etc. Hence, the technology adoption skills of a project manager will define the quantum of value addition, s/he brings in that project.

3. Risk Management

Big data projects need to have a very agile way of handling the processes. Due to the high amount of uncertainties involved it becomes a challenge to continuously map the expectations and deliveries. Some of the main apprehensions while leading a big data based project can revolve around-

i. The possibility that the data provided may fall short of providing analytical results in solving the required business problem. The estimates and plans done can only give you an approximate idea and are completely dependent on the behavior of the data; eventually these can be quite off target.

ii. Analytical results based on a large amount of data always carry the risk of the results being overly data dependent and not applicable across business scenarios. Significant changes in data points can trigger an altogether different or counter intuitive result.

iii. With the involvement of several stakeholders in a big data based project viz. solution vendor team, infrastructure provider, client etc. understanding the intricacies of data governance and data management policies can become a challenge.

4. Process Methodologies

For a big data project, the delivery methodology is mostly incremental and iterative. A lot of effort goes into the planning stage because of the high possibilities of failure during the proof of concept and proof of value stage. The process governing a big data project delivery needs to be flexible so as to accommodate more changes with the ever changing requirements of the clients. Extensive use of roadmaps, high level work breakdown schedules and milestones for progress measurement characterize such a project.

5. Big Data Analytics and Knowledge Management

A positive aspect of the huge amount of data in big data analysis is that it can be used to form a host of knowledge objects for future references and understanding. Integrating the analytical results and mapping them with the characteristics of the data set can help the solution providers to empower their clients with predictive models for designing their business roadmaps.

Thus we observe that a big data project is accompanied with several unique challenges and complexities. This makes it imperative for a project manager to cultivate specific competencies championing the delivery mechanism and thereby providing rapid deployment and cost effective business solutions.

References

Content