Converting IT pain to operational gain

Summary: Advances in technology are outpacing the adoption of new capabilities. The issue: Large companies can’t keep up with the changes and are stuck running legacy information technology architectures and old datacenter technology.

By John Rhoton for Transforming the Datacenter | April 11, 2013 — 16:05 GMT (09:05 PDT)

This content is produced by our sponsor Microsoft.  It is not written by ZDNet editorial staff.

Advances in technology are outpacing the adoption of new capabilities. The issue: Large companies can’t keep up with the changes and are stuck running legacy information technology architectures and old datacenter technology. The moving parts involved include:

• Budget
• Technical Transformation
• Architecture

Each of these has significant bearing on the future. We have been riding the leading edge of a technology wave that is changing how we work, the computing devices we use, and the back-end systems that support our operations, and how we adapt to that will separate winners from losers.

The changes have come quickly, and the innovations continue, often faster than big companies can respond. Much of this has happened during a slow economy and at a time when controlling costs is front and center on every IT manager’s to-do list. To further complicate things, government regulations are mandating long-term retention of data, which stresses capacity. Business managers are also demanding more sophisticated analyses of all this information to help them better market to customers. To top it off, ad hoc design changes generally lead to increased complexity over time, which increases the probability of error and triggers additional operational costs.

While these are not trivial matters, there is plenty of hope despite the challenges.

IT decision-makers can even achieve transformation to some extent with their current infrastructures for now, as long as there is some flexibility and agility baked in, and the platform is stable.

As the economy begins to improve and companies can no longer ignore the depth of technical transformation that has occurred in recent years, many executives have become impatient at the underperformance of their datacenters. While there are no quick fixes, the good news is that by taking a fresh look at IT strategy and the design of the datacenter, it becomes clear that datacenter transformation is possible.

Transformation is essential to remain competitive in today’s fast-moving world. The way I see it, the path forward is to not think in terms of patching the datacenter. Organizations instead need to assess their IT needs in the context of longer-term business objectives and act accordingly.

Let’s admit it: it is no small endeavor. Your datacenter needs to help you acquire and analyze the business intelligence you need to run smart operations. You need to be able to support a mobile workforce and communicate with mobile customers. You will also need to learn how best to manage those communications, as well as support and leverage machine-to-machine communications.

The silver lining is that addressing these needs is the first step in architecting the datacenter that meets your operational requirements and then you can think about what’s next.

Topic: Transforming the Datacenter

Where is the open #datacenter facility API ?

Posted Tue, 04/02/2013 - 00:03 by jan.wiersma

For some time the Datacenter Pulse top 10 has featured an item called ‘ Converged Infrastructure Intelligence‘. The 2012 presentation mentioned:

Treat the DC infrastructure as an IT system;

- Converge in the infrastructure instrumentation and control systems

- Connect it into the IT systems for ultimate control

Standardize connections and protocols to connect components

With datacenter infrastructure becoming a more complex system and the need for better efficiency within the whole datacenter stack, the need arises to integrate layers of the stack and make them ‘talk’ to each other.

This is shown in the DCP Stack framework with the need for ‘integrated control systems’; going up from the (facility) real-estate layer to the (IT) platform layer.

So if we have the ‘integrated control systems’, what would we be able to do?

We could:

• Influence behavior (can’t control what you don’t know); application developers can be given insight on their power usage when they write code for example. This is one of the needed steps for more energy efficient application programming. It will also provide more insight of the complete energy flow and more detailed measurements.
• Design for lower level TIER datacenters; when failure is imminent, IT systems can be triggered to move workloads to other datacenter locations. This can be triggered by signals from the facility equipment to the IT systems.
• Design close control cooling systems that trigger on real CPU and memory temperature and not on room level temperature sensors. This could eliminate hot spots and focus the cooling energy consumption on the spots where it is really needed. It could even make the cooling system aware of oncoming throttle up from IT systems.
• Optimize datacenters for smart grid. The increase of sustainable power sources like wind and solar energy, increases the need for more flexibility in energy consumption. Some may think this is only the case when you introduce onsite sustainable power generation, but the energy market will be affected by the general availability of sustainable power sources also. In the end the ability to be flexible will lead to lower energy prices. Real supply and demand management in the datacenters requires integrated information and control from the facility layers and IT layers of the stack.
• …

Gap between IT and facility does not only exists between IT and facility staff but also between their information systems. Closing the gap between people and systems will make the datacenter more efficient, more reliable and opens up a whole new world of possibilities.

This all leads to something that has been on my wish list for a long, long time: the datacenter facility API (Application programming interface)

I’m aware that we have BMS systems supporting open protocols like BACnet, LonWorks and Modbus, and that is great. But they are not ‘IT ready’. I know some BMS systems support integration using XML and SOAP but that is not based on a generic ‘open standard framework’ for datacenter facilities.

So what does this API need to be ?

First it needs to be an ‘open standard’ framework; publicly available and no rights restrictions for the usage of the API framework.

This will avoid vendor lock-in. History has shown us, especially in the area of SCADA and BMS systems, that our vendors come up with many great new proprietary technologies. While I understand that the development of new technology takes time and a great deal of money, locking me in to your specific system is not acceptable anymore.

A vendor proprietary system in the co-lo and wholesale facility will lead to the lock-in of co-lo customers. This is great for the co-lo datacenter owner, but not for its customer. Datacenter owners, operators and users need to be able to move between facilities and systems.

Every vendor that uses the API framework needs to use the same routines, data structures, object classes. Standardized. And yes, I used the word ‘Standardized’. So it’s a framework we all need to agree up on.

These two sentences are the big difference between what is already available and what we actually need. It should not matter if you place your IT systems in your own datacenter or with co-lo provider X, Y, Z. The API will provide the same information structure and layout anywhere…

(While it would be good to have the BMS market disrupted by open source development, having an open standard does not mean all the surrounding software needs to be open source. Open standard does not equal open source and vice versa.)

It needs to be IT ready. An IT application developer needs to be able to talk to the API just like he would to any other IT application API; so no strange facility protocols. Talk IP. Talk SOAP or better: REST. Talk something that is easy to understand and implement for the modern day application developer.

All this openness and ease of use may be scary for vendors and even end users because many SCADA and BMS systems are famous for relying on ‘security through obscurity’. All the facility specific protocols are notoriously hard to understand and program against. So if you don’t want to lose this false sense of security as a vendor; give us a ‘read only’ API. I would be very happy with only this first step…

So what information should this API be able to feed ?

Most information would be nice to have in near real time :

• Temperature at rack level
• Temperature outside of the building
• kWh, but other energy related would be nice at rack level
• warnings / alarms at rack and facility level
• kWh price (can be pulled from the energy market, but that doesn’t include the full datacenter kWh price (like a PUE markup))

(all if and where applicable and available)

The information owner would need features like access control for rack level information exchange and be able to tweak the real time features; we don’t want to create unmanageable information streams; in security, volume and amount.

So what do you think the API should look like? What information exchange should it provide? And more importantly; who should lead the effort to create the framework? Or… do you believe the Physical Datacenter API framework is already here?

More:

Good API design by Google : http://www.youtube.com/watch?v=heh4OeB9A-c&feature=gv

Original Article: http://datacenterpulse.org/blogs/jan.wiersma/where_open_datacenter_facility_api

A Greener Field: SDNs: Love 'em or Leave 'em?

Are Software Defined Networks (SDN) on your short list yet? Two recent surveys suggest it depends on where you work. I say that given our recent experiences with SDN, within five years and everyone will adopt or have definitive plans to adopt SDN technology. Here’s why:

SDNs, for those who’ve been sleeping with the pinecones of late, move intelligence previously locked within proprietary switching and routing hardware into open software. To put that in “networkese,” we separate the control plane from the forwarding plane using a protocol like OpenFlow. As such, SDN delivers on eight benefits:

• Agility. OpenFlow-based SDNs create flexibility in how the network is used, operated, and sold. The software that governs it can be written by enterprises and service providers using ordinary software environments.
• Speed. SDNs promote rapid service introduction through customization, because network operators can implement the features they want in software they control, rather than having to wait for a vendor to put it into plans for their proprietary products.
• Cost Savings. Software Defined Networking lowers operating expenses and results in fewer errors and less network downtime because it enables automated configuration of the network and reduces manual configuration.
• Better Management. OpenFlow-based SDN enables virtualization of the network, and therefore the integration of the network with computing and storage. This allows the entire IT operation to be governed more sleekly with a single viewpoint and toolset.
• Planning. OpenFlow can be easily integrated with computing for resource management and maintenance.
• Focus. OpenFlow-based SDN can better align the network with business objectives.
• More Competition. As a standard way of conveying flow-table information to the network devices, it fosters open, multi-vendor markets.
• Table Conversation. Instead of saying you working in networking at the next party, now you can say “I program the enterprise.” Now, how cool is that?

It’s no wonder in that a recent survey of service providers, Infonetics Research found that 80 percent of survey respondents are including OpenFlow in their purchase considerations. Rapid delivery of new services is essential for service providers and it’s a key benefit to SDN.

"While many uncertainties surround SDNs due to their newness, our survey confirms that the programmable network movement is real, with a good majority of service providers worldwide considering or planning purchases of SDN technologies to simplify network provisioning and to create services and virtual networks in ways not previously possible,” notes Michael Howard, co-founder and principal analyst for carrier networks at Infonetics Research.

On the more “curmudgeonary” side of life you have Mike Fratto noting that enterprises haven’t yet gone gaga over the whole SDN thing. “Enterprises aren't really ready for SDN quite yet, as the results from a recent InformationWeek survey of 250 IT professionals showed. Some 70% of respondents said they weren't even going to start testing SDN for at least a year.”

So which is it for you? Are you a service provider kind of guy or an enterprise kind of gal?

via networkworld.com

Virtualization of Data Centers: New Options in the Control & Data Planes (Part II)

RAGHU KONDAPALLI
LSI Corp.

This Industry Perspectives article is the second in a series of three that analyzes the network-related issues being caused by the Data Deluge in virtualized data centers, and how these are having an effect on both cloud service providers and the enterprise. The focus of the first article was on the overall effect server virtualization is having on storage virtualization and traffic flows in the datacenter network. This article dives a bit deeper into the network challenges in virtualized data centers as well as the network management complexities and control plane requirements needed to address those challenges.

Server Virtualization Overhead

Server virtualization has enabled tens to hundreds of VMs per server in data centers using multi-core CPU technology. As a result, packet processing functions, such as packet classification, routing decisions, encryption/decryption, etc., have increased exponentially. Because discrete networking systems may not scale cost-effectively to meet these increased processing demands, some changes are also needed in the network.

Networking functions that are implemented in software in network hypervisors are not very efficient, because x86 servers are not optimized for packet processing. The control plane, therefore, needs to be scaled somehow by adding communications processors capable of offloading network control tasks, and both the control and data planes stand to benefit substantially from hardware assistance provided by such function-specific acceleration.

The table below shows the effect on packet processing overhead of virtualizing 1,000 servers. As shown, by mapping each CPU core to four virtual machines (VMs), and assuming 1 percent traffic management overhead with a 25 percent east-west traffic flow, the network management overhead increases by a factor of 32 times in this example of a virtualized data center.

Click to enlarge chart.This table shows the effect on network management overhead of virtualizing 1,000 servers.

Virtual Machine Migration

Support for VM migration among servers, either within one server cluster or across multiple clusters, creates additional management complexity and packet processing overhead. IT administrators may decide to move a VM from one server to another for a variety of reasons, including resource availability, quality-of-experience, maintenance, and hardware/software or network failures. The hypervisor handles these VM migration scenarios by first reserving a VM on the destination server, then moving the VM to its new destination, and finally tearing down the original VM.

Hypervisors are not capable of the timely generation of address resolution protocol (ARP) broadcasts to notify of the VM moves, especially in large-scale virtualized environments. The network can even become so congested from the control overhead occurring during a VM migration that the ARP messages fail to get through in a timely manner. With such a significant impact on network behavior being caused by rapid changes in connections, ARP messages and routing tables, existing control plane solutions need an upgrade to more scalable architectures.

Multi-tenancy and Security

Owing to the high costs associated with building and operating a data center, many IT organizations are moving to a multi-tenant model where different departments or even different companies (in the cloud) share a common infrastructure of virtualized resources. Data protection and security are critical needs in multi-tenant environments, which require logical isolation of resources without dedicating physical resources to any customer.

The control plane must, therefore, provide secure access to data center resources and be able to change the security posture dynamically during VM migrations. The control plane may also need to implement customer-specific policies and Quality of Service (QoS) levels.

Service Level Agreements and Resource Metering

The network-as-a-service paradigm requires active resource metering to ensure SLAs are maintained. Resource metering through the collection of network statistics is useful for calculating return on investment, and evaluating infrastructure expansion and upgrades, as well as for monitoring SLAs.

The network monitoring tasks are currently spread across the hypervisor, legacy management tools, and some newer infrastructure monitoring tools. Collecting and consolidating this management information adds further complexity to the control plane for both the data center operator and multi-tenant enterprises.

The next article in the series will examine two ways of scaling the control plane to accommodate these additional packet processing requirements in virtualized data centers.

http://www.datacenterknowledge.com/archives/2012/08/20/virtualization-of-data...

VMware to buy Nicira for $1.26B in a strategic leap of faith — Cloud Computing News

Nicira’s CEO Martin Casado

VMware will shell out $1.26 billion in cash and an assumption of existing equity to buy Nicira, a company that has built software to do for networking what VMware has done for virtualizing computing. That’s a lot of money, especially when you consider how few production-level implementations there are of the software-defined networks that Nicira is building.

Ncira was created five years ago and has raised $50 million from investors that included Diane Greene, an original founder of VMware. It makes controller software that helps free the act of moving data and packets around a network from the constraints of networking hardware  – an increasingly tough problem inside highly virtualized and webscale data centers. As I explained in a story on its launch (it was titled, “Meet Nicira. Yes, people will call it the VMware of networking”):

Nicira is one of several companies attempting to solve the problem that Greene helped create when she co-founded VMware to push hypervisors and virtualization. Once servers were virtualized, it created an easy way to separate computing from the physical infrastructure. The benefits of server virtualization were more-agile compute infrastructures — a developer would spin up a server in minutes as opposed to waiting days for approvals — as well as consolidating IT. Storage followed, but holding the whole virtualized infrastructure effort back was networking. Like a bird with its wings clipped, IT was tethered to the physical hardware by networking.

Nicira has plenty of paying customers including eBay, AT&T, Rackspace and NT, but it’s unclear how many of these are running Nicira’s controllers in their production environment. With this purchase, my sense is that VMware is paying big money to get in on this space because its collaborations with Cisco have failed to deliver on a true solution for many of its customers, and it can’t afford to be left behind as the wave of interest and eventual spending on software-defined networking hits a peak.

In a blog post covering the deal, VMware CTO Steve Herrod explains that Nicira will fit in with VMware’s vision of a software defined data center — although the specifics of that vision have yet to be unveiled. And it’s true that if we consider how Nicira customer Calligo is using the Nicira controller, it fits within what I would think of as a software-defined data center — essentially a pool of compute resources that are tied together with software and where the physically hardware can reside in different data centers and pulled up for use as needed. The whole process is programmable and automated.

In pushing this model, VMware was always going to have to sign partnerships to deliver the software-defined networking elements, because its own VXLAN option was more of a an encapsulation scheme than a true separation of the networking logic from the hardware pushing the bits. By buying Nicira, VMware has chosen a software-defined networking vendor that plays with open protocols such as Open Flow, but is still focused on keep a proprietary edge to its business. That’s a similar strategy that VMware seems to be pursuing as well.

The deal is expected to close during the second half of this year. The deal includes approximately $1.05 billion in cash plus approximately $210 million of assumed unvested equity awards.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Migrating media applications to the private cloud: best practices for businesses
• Infrastructure Q3: OpenStack and flash step into the spotlight
• A field guide to cloud computing: current trends, future opportunities

via gigaom.com

Olympic legacy to live on though data center | Datacenter Dynamics

The London Legacy Development Corporation announced another win for East London’s technology dreams, which this time includes a large data center at London’s Olympic Park.

UK data center company Infinity is one of the joint venture members behind the project.

Once the athletes and spectators have vacated the site, which will be renamed the Queen Elizabeth Olympic Park, workers from the JV called iCITY will move in to turn the Press and Broadcast Centre into what they say will become a world-class center for innovation and enterprise.

They claim the project will create 4,600 direct jobs on site as start-ups, media companies, a university, digital academy, restaurants, bars and more move in.

A further 2,000 jobs are expected to be created through the supply chain and consumer spending.

iCITY is a joint venture between real estate investment and advisory company Delancey and Infinity SDC.

Infinity will build a data center that will be between 250,000 to 350,000 sq ft data center with 90MW capacity designed for low-latency, high capacity data center needs.

It will sit on what is currently one of the most digitally connected buildings in the world, set up as a 24-hour media hub for more than 800 journalists covering the Games.

The building was built in ten weeks and includes state-of-the-art utilities, power and digital connectivity, 52,000 sq m of studio space over two double-storey floors.

It also has its own 200m long High Street with banks, travel agents and other facilities, a Transport mall and a 12,000 sq m catering village.

The building’s 2,500 sq m roof is flat and covered with concrete, gravel and moss and logs to encourage local invertebrates and other local wildlife

Infinity SDC CEO Martin Lynch said the new data cener will help the company move into new London markets around digital media but will also target large investment banks, insurers, international  brokerages, telecommunications companies and outsources.

“This is a strategically important move for Infinity that sees us expand into a new sector,” Lynch said.

“The data center at the heart of this development will ensure low latency, high capacity and streaming environments that are ideally suited to animation, post production and broadcasting.”

The park will be open in separate phases from 27 July 2013, following the removal of temporary venues, the building of new roads and bridges and the building of its own neighbourhood.

 

via datacenterdynamics.com

The 3 Common Pitfalls of #DCIM Data Center Infrastructure Management | #DataCenter

via blog.schneider-electric.com