Running Fedora 14 on my netbook, I needed more space for the root directory (in the listing below, in bold, the space available, the space utilized and the remaining space). I use a logical volume manager (that is an abstraction layer above the hard drives and helps manage them flexibly – HP-UX was the first Unix to use LVM, by the way) as well as luks for the encryption
[root@netbook2 herve]# resize2fs /dev/dm-3
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/dem-3 is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/dm-3 to 1048071 (4k) blocks.
The filesystem on /dev/dm-3 is now 1048071 blocks long.
Checking if everything went well (in bold, the size has increased):
As I am working on Java benchmarks on a DL980 on the top of VMware and RHEL, I needed to execute the same script on 32 different virtual machines. I could have written an bash script to do so, but I have found a very nice suite of tools that does the job (and more) called parallel-ssh.
This Google Summer of Code project written in Python allows to execute a SSH command, to copy files, to rsync directories and to kill processes (among others) on multiple hosts
The install
First download the .tar.gz document
# wget http://parallel-ssh.googlecode.com/files/pssh-2.2.2.tar.gz
untar it
# tar -xzvf pssh-2.2.2.tar.gz
# cd pssh-2.2.2
To install (you need Python, it worked on RHEL 5.3):
# python setup.py install
The configuration
Then to execute parallel commands, create a file with all IP/hostnames you
need (one per line) like this
# cat hosts.txt
192.168.10.1
192.168.10.2
Create the directories “stderr” and “stdout”
# mkdir /tmp/stderr
# mkdir /tmp/stdout
and that’s it !
How to use it ?
If you want to execute the command hostname ?
# pssh -A -e errors -o /tmp/stdout -e /tmp/stderr -h hosts.txt -l root hostname
-A will ask the root password inline
-o will put the standard output in one file per host in the directory /tmp/stdout
-e will put the standard errors in one file per host in the directory /tmp/stderr
-l is the user that will execute the command
For my benchmark I nedded to change the starting script run.sh, so I edited it on one virtual machine and then used pscp to copy it on all the nodes in the directory /root/java/
# pscp -A -h hosts.txt -l root /root/java/run.sh /root/java/
And finally I executed the benchmark on my 32 nodes
# pssh -A -e errors -o results -h hosts.txt -l root /root/java/run.sh
Have you forgotten one JVM garbage collection parameter ? No worries, pnuke will make sure that all java processes are stopped on all servers
# pnuke -A -h hosts.txt -l root java
This post is the second of a series of three that will explain the concepts and technologies that are used in HP CloudSystem Matrix. The first was about creating a CloudMap. This post is about the provisioning and automated deployment of an IT application. In a third post, I will address how to manage the pools of resources !
CloudSystem Matrix is a private cloud solution that aims at speeding the deployment of applications through highly automated technologies and integrated processes.
After we have created and published our cloud map, we have provided our business users all the information they need in order to deploy the application they require to execute their company’s strategy.
In order to do so, the user logs in to the self-service portal.
As you can see, the home page shows which resources are put at disposal to the user, as well as the current service consumption. We see that five physical servers, out of the six at disposal, as well as two virtual machine hosts are in use as part of a service deployed by the CloudSystem Matrix. By clicking on the “Templates” tab, we can access the list of templates which are published and which the user has the right to deploy. Remember the SAP 2-tier template we created in the first part? We have published it, hence we find it back here. That the user has created the template or not, doesn’t play a role as long as he has the right to use it.
By clicking on the template, the business user can display the details of the service: the layout of the network, the price, as well as the notes entered by the creator of the template, etc. In order to deploy the service automatically, he will click on the button ‘Create service’.
The user now has to enter a service name (for instance “SAP 2-tier John Smith”), the hostname completion (remember the need for a hash sign in the first post ?), an email -to contact the service requestor-, a start date and hour at which the service will be deployed, an end-date, at which the service will be rolled back, and finally, a pool of resources, from which the resources, such as servers, storage and networking, will be picked from. Final step: he clicks on “Submit”, and here we go !
The request is sent and the activity which is happening behind the scenes is displayed: the resources are reserved from the pool of resources. The software makes sure that the pool can accommodate the need indicated in the template. We can see here that four servers, four IP addresses, three networks and four boot disks are requested. CloudSystem Matrix will pick from the pool of resources the servers that meet at least the requirements of the template and will pick from storage disks (LUNs) that are already presented to WWN (worldwide names) and also meet the size indicated.
We see here that CloudSystem Matrix also integrates nicely into the user’s processes: the request above is paused, because an administrator (or an IT manager or anyone else who should take this decision) must give his approval, so that the deployment can continue. A Request for Change (RFC) could also be triggered at this point in time with tools such as HP Service Manager or BMC’s remedy. The teams using these tools will just have to accept the request and the deployment will go on.
We see here that the administrator granted the authorization to deploy this service and that the allocation process has started. The requested resources are then provisionned. The physical servers are reserved and a custom HP Virtual Connect profile is applied to them. This means that we are applying one ore more MAC addresses for the network, which will be connected to the VLANs we need, as well as one or more WWN for the storage, to which disks are already presented. The zoning of the storage was already prepared, so that after the WWN is attributed to the server, it can install the operating system as well as the application straight after the VC profile was attributed.
Taking a look at the Onboard Administrator, the HP BladeSystem enclosure management processor, the server loses the “i“-icon, which means that the server has now a Virtual Connect profile
On the screenshot above, we can see “Provisioning the logical server” (either physical or virtual) and, finally, “Activating logical server”: the ProLiant or Integrity server is now starting.
If VMware or Hyper-V virtual machines were to be deployed, CloudSystem Matrix would pick the right template, clone it, provision it, and finally adapt it according to the needs of the cloud map. If HP Integrity Virtual Machine were deployed, the host would be contacted vis SSH, it would create the VM and deploy it with Ignite UX either through normal package deployment, or via a golden image, a concept similar to the VMware/Hyper-V template.
We see above that the Integrity server starts, it then boots from the network (since the disks we have presented are empty) and that Ignite UX detects a deployment request from the client.
All the deployments methods (Ignite-UX, HP RDP, HP Server Automation, etc.) and CloudSystem Matrix can communicate with each other: the deployment server confirms to the Matrix operating environment that the deployment request of a given client was received. This is acknowledged by the management server which indicates it to the user.
Finally, after the operating system is deployed, the application is installed through the execution of a workflow (it will be the subject of another blog post in the future).
Although I discussed the technical details of what happens behind the scenes, the business users, when they make a request, can see a completely abstract high-level view of what is going on. They don’t need to know the details, they just want to know if the application was deployed.
On the picture above, the business user sees that everything went well and that his IT service is up and running. What could have taken so much time to deploy, lasted roughly one hour with HP CloudSystem Matrix.
Instead of taking months putting all the pieces together, CloudSystem Matrix orchestrates the provisioning of storage, server and networking, installs the operating system and deploys the application in an automated way, also integrating in the organizations’ processes.
It is a fantastic tool that was already embraced by customers to modernize their IT environments. I have personally helped some of them deploying this solution in their datacenters and they love it. Though it needs a good understanding of key HP technologies (HP BladeSystem, HP Virtual Connect, HP Systems Insight Manager, etc.), the HP services team take care of the deployment of the solution, and the complexity of the infrastructure is hidden to the business users.
In my third and last blog entry for this series, I will focus around managing the infrastructure to fill the different pool of resources and how to present them to the users.
This white paper written by Forrester and commissioned by HP, highlights the reasons why CIOs working in different industries continue to invest in Unix operating systems from the three major vendors (HP, IBM and Oracle). These individuals are confronted with a high pressure to reduce the costs and successfully complete the projects they start. I remember my software development class at the HTW Saarbrücken, during which our professor told us that half of all IT development projects fail, let alone the implementation and the move to production !
During the last past years, the rise of Linux-based operating systems, mostly on x86 architectures, was spectacular, because distributions such as Red Hat were able to provide a lower-cost alternative in terms of acquisition. However, the vast majority of these decision-makers, who, besides, also embraced Linux in their datacenters, thinks that Unix-based operating systems still bring more value in the following categories:
- availibility
Indeed, when it comes to availability, Unices have an advantage. The maturity of the product is higher, but above all, the integration of the platform to the operating system and to the application give an indisputable advantage to RISC/EPIC-based servers. The clustering solution ServiceGuard, for instance is way ahead of what SuSE HA or the Red Hat Cluster Suite can dream to become in 2 years. Though it is possible to reach high levels of availability for the database stack on Linux with Oracle RAC (Real Application Clusters), the management of the solution is quite tedious and its cost prohibitive.
- scalability
Clearly, the biggest deployments of very large databases or ERPs happen on Unix. However, Linux is catching up very fast. Red Hat 6 supports the 120 logical CPUs and 4TB of RAM of the brand new HP DL980 ! Especially when it comes to scalability with regards to virtualization for massive consolidation scenarios, Linux does an excellent job with the compelling argument of the full binary compatibility with the x86_64 architecture.
- ease of maintenance
The number of updates is far less frequent for Unix (at least for HP-UX that I know well) compared with Linux. Moreover, tools such as Dynamic Root Disk, which allows to relocate the operating system from the local disks to the SAN in one command line and one reboot, or to patch the OS on a mirror offline and to reduce the planned downtime to the reboot time, really increase the productivity of the admins. Once again, the tight integration of the operating system to the platform makes this happen.
- partitionning
The partitioning of the RISC-EPIC hardware platform is much more advanced. Because of the capabilities of their platform, x86-based servers rely mainly on hypervisor-based virtualization, whereas Unices offer both hard and soft partitions. Also, although IBM offers Linux on Power servers, most of the virtualization capabilities are only available for AIX, IBM’s Unix. Finally, Linux lacks the pay-per-use and workload management functions of Unices. cgroups implemented in RHEL6 are a beginning, but it still lags behind the HP SRP or Solaris Zones.
A personal note about the “Staffing and skills are not an issue”: my first system administration experience started with Linux and I learned HP-UX afterward. The main principles of *NIX, such as LVM, runlevels, init scripts, the Bash shell, etc. are the same and hence the lack of smart people able to learn a few more commands should never appear. The most important is to stay flexible and ready to learn new technologies !
To conclude, Unix is going to remain in the datacenters for long. Its stability, ease of maintenance and workload management ensure a great continuity in the IT service delivery. Though the number of Linux servers will continue to grow, driven by the increase in performance of the x86_64 platforms (and I am definitely fine with that !), HP-UX, AIX and Solaris (depending on Oracle’s good will) are here to stay…
As part of an exchange program within HP, I will move for three months to Cupertino, California to support a worldwide strategic presales team.
I am really thrilled to go there for three reasons:
- being in Cupertino is an extraordinary opportunity to meet all the people from HP’s engineering teams
- belonging to this worldwide strategic presales team is a great challenge and a move outside of my comfort zone in a foreign country. I love it
- the USA is a fascinating country and I hope to learn as much as possible from a cultural point of view. Attending Toastmasters meetings there should also be very interesting
I will report about my stay in California and about my findings and experiences here.
In this interesting and easy-to-understand presentation, Professor Arturo Bris from the IMD business school in Lausanne, Switzerland, explains the latest economic evolutions in the world:
- what is happening in the US from an economic perspective
- the effects of the quantitative easing on the economy and on the finance industry
- an analysis of the PIIGS (Portugal, Italy, Ireland, Greece, Spain) countries which encounter serious difficulties with their sovereign debt.
The key take-aways for me were:
- the ration debt to GDP that is shown in every explanation of the Euro crisis is not an efficient indicator to measure how endangered a country is with its own sovereign debt. It all depends on his capacity to reimburse his creditors.
- Germany benefits from the sovereign debt crisis of the PIIGS crisis, but Angela Merkel must be careful with the German public opinion not to give the impression to bail out the other countries’ mistakes with German taxpayers money… tough task !
One remark : the USA can afford to have a weak dollar to export, since the oil, which in industrialized countries plays a critical role in imports, is payed in dollars. They don’t have the currency effects that affect other countries. This system will be beneficial to them as long as investors trust the value of the dollar. Will it always be the case ?
Finally, I have one question: Professor Arturo Bris states that the PIIGS countries, since they are net importers (i.e. with a negative trade balance), have an interest in a strong euro. But if the main providers of Spanish imports are already in the Euro zone, a strong Euro would not make a difference. Wouldn’t it ? And wouldn’t a weak Euro help increase the inflation and hence reduce the debt burden that the PIIGS country have to carry ?
My parents have a quite old Packard-Bell laptop Easynote R1934. In order to help them increase the security of their computer, let them have a nice desktop experience and increase the Free software footprint, I have installed Ubuntu 10.04 on it. The only component that did not work is the wireless chip, which is a RaLink RT61, as shown here
This chipset has a Linux kernel module driver “rt61pci”, but this one did not work. Though iwconfig recognized the chip and ifconfig -a showed the wireless interface wlan0, the following command
famillemarcy@ubuntu-home:~$ lshw -C network
showed the wireless device as disabled.
In order to let it work, I had to add the following parameters in /boot/grub/menu.lst “pnpbios=off acpi=force irqpoll”. Only after having added it and rebooted it could work.
A colleague of mine and I were invited in Ramallah by a HP partner in Palestine to give presentations about the latest HP technology in servers and storage. This was an amazing four-days experience from which I learned a lot.
After I had seen the invitation for this event, my first reaction was to check on both the French and German ministry of foreign affairs to see if it was really safe to go there. Fortunately, both of them made clear that there was no risk of going to the West Bank, but that we should take to behave appropriately.
We arrived right from Istanbul after the HP Enterprise Technology & Solutions Summit at the Tel-Aviv Ben Gourion airport and were driven by our contacts directly to Ramallah.
The first thing I learnt from this visit is how hard the environment is for Palestinian businesses. Due to political reasons, there is a lack of support that hinders the development of the Palestinian economy. Freedom of movement to enter Israel from the Palestinian territories is restricted, and there are checks along the roads, either from Palestinian security forces or from Tsahal. Furthermore, the state of the roads is poor and increases the time spent to drive even a few kilometers to hours. The electricity delivery was, however, impeccable , as were the cell phone network coverage and the Internet connectivity in hotels.
From what I could experience during these four days, the Palestinian people have a lot of courage to develop new businesses in these tough conditions. Although the situation is calm in the West Bank, the feeling remains that everything could change from one day to another. There is a saying in Palestine “Anyone who succeeds to open a business in Palestine can do that anywhere in the world”. It seems to be quite true.
From a technology perspective, we were expecting to present rather lower-end solutions, such as ProLiant – x86_64 based – rack-mount servers, c-Class ProLiant blades and P4000 iSCSI solutions. We were astonished to see that some Palestinian customers were using high-end servers, such as Integrity or even NonStop servers !
We visited the headquarters of top customers to discuss their IT and business strategies, their projects and to visit their datacenters. Again, the technology used in their datacenters was top-notch (HP c-Class blade systems, HP Virtual Connect, NonStop servers, etc.) as were the datacenter layout and facilities. We were really impressed by how advanced these datacenters were and how well organized and clean the setup was.
Finally, a very interesting aspect of this customer visit was the cultural aspect of meeting customers and partners. The relationship network is an essential part of the business life. To do business, people need to be friends, or at least know each other.
Also, unlike in Germany, for instance, where touching people without their consent is a no-go, it is absolutely normal to do so in Palestine. Men kiss each other on the cheeks when they are friends and frequently backslap each other. People also have a very strong forward-looking mentality. They tend to avoid talking about the general situation of their country when they are not asked directly and rather focus on what is next going to happen to them in the future.
To conclude, this visit in Palestine was a great experience. Though the business environment is extremely tough, people are still focused on creating opportunities and succeeding in making money. The road infrastructure seems to be a pain point for the economy but others, such as the mobile phone or Internet connectivity are impressive. The technology used by customers and their datacenters we visited were good as their European counterparts and the cultural environment was extremely interesting.
I am looking forward to following up with opportunities in this country and doing business with people from Palestine !
This week, I received a new smartphone from my employer, HP. Following the acquisition of Palm and instead of my old iPaq with Windows Mobile which quality was low, to say the least, the HP Palm Pre 2 was offered to HP employees, provided they have the consent – and funding – of their manager.
What I can say, so far, is that I love this phone ! As already claimed by many bloggers, webOS is brilliant and very intuitive. I can synchronize my emails, calendar and contacts from Exchange. On the top of that, although a proprietary operating system, webOS is based on the Linux kernel, which makes it even more attractive for me and, I think, will make HP an even more important contributor to the Linux ecosystem.
One of the greatest features of this phone is definitely the Touchstone. This station allows to load the phone’s battery through electro-magnetic fields just by putting the smartphone on the station, without having to plug any cable !
I see everyday HP employees from my LinkedIn network activating their LinkedIn application for webOS. I think it is a good thing for the platform to make it attractive for developers. HP’s strategy to make webOS the reference operating system for embedded devices, such as printers, phones, etc. is the right one and I hope to see many more applications for my phone in the future.
This video is not from me but shows how the Palm Pre 2 and webOS look like. Try it and get it !
This post is the first of a series of three that will explain the concepts and technologies that are used in HP CloudSystem Matrix. This post is first about the creation of a cloud map. The second will present the deployment of a complete service and the last will show how to manage the pools of resources !
When I ask my customers how long it takes them to deploy a complete IT service (i.e. architecture design, proof of concepts, physical reception of the servers, provisioning of the resources -such as storage and networking- install the operating system and configure the application) I generally hear answers going from a couple of weeks for small projects to months for more complex projects.
This is the main infrastructure issue the HP CloudSystem Matrix addresses. This Matrix is simply a complete solution made of hardware (c7000 blades enclosure and a wide choice of Intel Itanium-based Integrity or AMD Opteron or Intel Xeon based servers), software (the full stack of the so-called Insight Software Suite) and services (the HP services people come on site, install the hardware and software and customize the solution to adapt it to the customers’ needs and processes).
Instead of deploying applications the old way, HP Bladeystem Matrix really transforms the hardware in commodity hardware that can be used and reused by different services. It creates a shared-service infrastructure that can be divided in different pool of resources which users will pick in order to request a service to be deployed. We can call this a cloud infrastructure in a box, because the users request applications through a self-service portal and these applications are deployed automatically somewhere, on an highly scalable IT infrastructure the business user has no idea about – and that is not his job anyway !
To be concrete: everything starts with the Insight Orchestration Designer. In this web-based application, it is possible to create so-called “cloud maps”, or templates, which will contain all the important information to deploy IT services. Let’s say you want to deploy a SAP 2-tier infrastructure: you will need a database server and some application servers, all booting from the SAN for more flexibility, as well as some network connectivity, i.e. VLANs that will be help deploying the application and the operating system over the network through PXE-boot and also, most importantly, a network linking the application servers to the database server.
How long and in which time frame do you think your company will be able to deploy all these elements manually and individually ? If you think what you have today is not good enough, HP CloudSystem Matrix can help you.
You can create your own cloud map or pick the maps already available on the HP website. The advantage of these pre-prepared cloud maps ? They are designed according to the best practices from HP and the ISVs, such as Oracle, SAP or Red Hat, so you can be sure that you will have enough I/O resources, for instance, to accommodate the deployment of a database server.
On this example, you see the two deployment networks for PXE-boot (RDP for HP Rapid Deployment Pack for Linux on x86-64 and Ignite-UX for HP-UX). The elements, such as server groups or physical storage are picked from the top left corner, drag-and-dropped on the main surface and connected to each other. Creating such a cloud map is really easy, however, this is just a XML file, there is no on-the-fly deployment.
Then, each of the components, such as servers, storage or networking must be configured to enter the minimum requirements needed. At deployment time, the software will pick the elements from the pool of resources which meet at least these requirements. In the case below, we see that the database server will at least have two processor cores and 8GB of RAM.
Note also that you can enter the cost for physical and virtual servers, IP addresses and GB of storage. This helps give an idea of the cost of an IT service to business users. This cost can also be integrated into other applications through a REST API.
We then configure the two network interfaces defined for the HP-UX database server. The primary interface will be the one deploying the operating system (the Ignite-UX network). We then have the choice between a DHCP, static or “Automatic” IP address.
DHCP is the standard dynamic host configuration protocol and “static” will allocate a static IP address to the interface. The trade-off however, is that we won’t be able to re-use the cloud map into multiple services, since the IP address will already be taken.
The field “Automatic” resolves this problem in that the management software will take an IP address from a pre-defined range and will allocate it as a static IP address for the duration of the service. The server, when it restarts, for instance, will keep the same static IP address. When the service is erased, the “Automatic” IP address will be sent back to the pool and can be reused for other services.
Finally, the hostname must contain a hash (#) sign. The reason is that, again, this template can be deployed multiple times and the hostnames need to be different at every deployment. At deployment time, the #-sign will be replaced by a completion string given by the service requester.
We now have to select the operating system that will run on the selected server. In our case, we use HP-UX 11i v3. The x86_64 servers can use Red Hat Enterprise Linux or Windows. Virtual machines running on the top of VMware or Microsoft Hyper-V can also be included in these cloud maps and be provisioned and deployed automatically.
After the main characteristics of the server were defined, we can now make use of the workflows to finish the cloud map. In our case, the workflow “SAPInstall” will be executed at the end of the creation of the service. This means that the SAP instance will be deployed on our newly provisioned infrastructure.This technology is based on HP Operations Orchestration and allows to customize the deployment of the service down to the smallest detail (e.g. change of a kernel parameter,etc.)
At that point in time, it is possible to create your own workflows, such as an integration with HP’s Universal CMDB to update it with the service, or open an new Request for change (RfC) in HP Service Manager or with BMC’s Remedy. The customers can hence integrate the deployment of their services in their already existing ITIL change processes.
Our cloud map is now ready to be deployed. The logo next to “validation status” is now green, which means that we have enough informations to deploy this service. We now save it and tick the box “Published”, so that business users will be able to select this cloud map from their service catalogue and request it to be deployed automatically.
The creation of a cloud map was an easy thing. Remember, though, that HP, in co-operation with ISV’s provide cloud maps ready to be deployed. In my next post, I will show you how to request a service to be deployed and what happens behind the scenes. Stay tuned !
Technology and leadership in an international environment
