Search | User Listing Vidsite | Calendars | Quotes
Home Page ->  General Discussion -> Operating System & Software -> View ThreadLogon (or Register, or Join TradeTight)

You are logged in as a limited-access Guest.To join TradeTight, first read the info in the Organization & Content room, then click the link above. 

James 1:4 (AMP) ... But let endurance and steadfastness and patience have full play and do a thorough work, so that you may be perfectly and fully developed, lacking in nothing.


Sticky Virtualize a Computer & Other Quick Tips
Jump to page : 1
Now viewing page 1 [50 msgs/pg]
Jump to room :
JeffBrown
Posted 5/21/2018 5:46 PM (#9089)
Subject: Virtualize a Computer & Other Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
How to copy a physical machine and make it virtual
Procedure-PDF attached to end of post

So, you have a laptop that has OT installed on it and you just got a shiny new laptop. You don’t want to go through the process of setting up OT all over again on the new laptop. Wouldn’t it be nice if you could just virtualize the old laptop? Most often, you can. This process is called P2V (physical to virtual).

There are multiple ways you can convert a physical computer to a virtual, but all methods essentially read the disk of your physical computer and create a new virtual disk. You then attach this virtual disk to a VM and you have a virtualized version of your old laptop. This works most of the time. Some things may get in the way, such as hard disk encryption and specialty hardware, but most physical computers can be virtualized.

I’ll walk through the high level process. This process works regardless of your hypervisor (VMware, VirtualBox, or even Hyper-v). The most important aspect of this migration is the format of the disk file (VDI, VMDK, or VHD). VirtualBox supports all three formats while VMware prefers VMDK and Hyper-v wants VHD. I’ll also cover how to convert a file from one format to another using a VirtualBox utility.

Step 1: Choose your software to virtualize the disk.
Here I’m going to point to a couple of sites that provide different options available for Windows. Any will work. Some are free, others may be expensive. Ultimately it is your choice. I prefer free.

Backup Method
Disk2VHD Basics from Microsoft
Disk2VHD Process
Clonezilla Alternative

The processes are similar. You essentially want a software product that will backup your complete hard disk and place it in a virtual disk file. Disk2vhd will do this directly and can be run on a running Windows instance. I have not used this utility but it is reported to work well. The output is a vhd format virtual disk, ready to be attached to a Hyper-v VM. By the way, Hyper-v is Microsoft’s hypervisor and is included with Windows 10 Pro, Enterprise, or Education version. It is not available on the Windows 10 Home edition.

Step 2: Conduct your transformation from Physical to Virtual
Before proceeding with either of the 2 options below, it is a good idea to remove any software that you don’t use or don’t want in the new virtual machine. Spending a little time cleaning up your old laptop will help keep the image small and not bloated with unused software. Make sure to empty your trash and run disk cleanup to get rid of all the garbage that has likely accumulated over the years. Certainly keep what you want, but the less you have installed, the quicker the conversion will go.

Option 1: Using Disk2vhd.
• Run Disk2vhd on the old computer and create the vhd file.
• Transfer the file to the new host computer.
• Create a new virtual machine on the host computer using your hypervisor of choice.
• Attach the vhd file to your virtual machine as its main disk. If your hypervisor can’t use a vhd file, look below for how to convert it.
• Done.

Option 2: Using the Backup/Restore approach.

If you decide to use a backup/restore approach with either AOMEI Backupper or Clonezilla, there are just a few more steps.
• Conduct the backup of your old computer. Depending on the software, you will either need to boot up the old computer using the software product (Clonezilla method) or install the backup software and create the backup (AOMEI method). With both methods, you have effectively backed up the old computer. You need an external storage device such as a big USB drive.
• Restore the backup image to the new VM. In both cases, you will need to boot the VM with the software provided by the backup provider. That means if you used Clonezilla to backup the old laptop you need to boot the VM using Clonezilla. Or if you used AOMEI, you need to use its special bootable program to start the VM. These are usually available as a CD or USB which you attach to the VM and set the boot order to look at those devices first. You then conduct the restore.

Hey, great, I used Disk2vhd but really want a VDI or VMDK file. Now what?

So, you went with the simple approach but instead of using Hyper-v, you want to use VirtualBox or VMware. Even though VirtualBox can use the vhd file, you may want to convert it to VDI format or VMDK format for performance or consistency reasons. Luckily, VirtualBox has a utility to handle this conversion.

• If you haven’t installed VirtualBox, this is the time to do so.
• Now go the the command line (Right click the Windows Start button and select Windows PowerShell (Admin)). Sorry, but you need to run this from the command line.
• Use the cd command to navigate to the directory where you have your vhd file. If you need help navigating on the command line, google is your friend. Once you are in the directory where the .vhd file is located, issue the following command:
“C:\Program Files\Oracle\VirtualBox\VBoxManage.exe” clonehd vm1.vhd vm1.vdi --format vdi

• Attach the vdi or vmdk file to your new VM as its main disk.

NOTES:
• You need to have quotes around “C:\...VBoxManage.exe” since the path contains a space.
• Replace vm1.vhd with the file name of your vhd file and vm1.vdi with the name you want for the vdi file.
• Also, if you want the format to be vmdk, substitute vmdk for the format instead of vdi.
• For more information on the VBoxManage command and clonehd, see the help manual of VirtualBox.


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer


Attachments
Attachments VirtualzeAPhysicalComputer.pdf (43KB - 4 downloads)
Top of the page Bottom of the page
JeffBrown
Posted 5/21/2018 5:46 PM (#9090 - in reply to #9089)
Subject: VirtualBox Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
Quick Tips when working with a VM under VirtualBox
Attached PDF has the contents of the Quick Tips

Virtual Machines are easy to use, but each hypervisor works a little different. Spending a little time getting familiar with the different features will be helpful to get the most out of your virtual experience. Here are just a couple of quick, easy features that I’m sure you will find useful.

• Click the window close button (the X in the upper right of the VM window) and you’ll get a pop-up.

(Close-1.png)




Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer


Attachments
Attachments Close-1.png (18KB - 0 downloads)
Attachments VirtualBoxQuickTips.pdf (589KB - 3 downloads)
Top of the page Bottom of the page
JeffBrown
Posted 5/21/2018 5:46 PM (#9091 - in reply to #9090)
Subject: VirtualBox Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
• Saving the machine state is similar to suspending a physical computer. When you save the machine state, it starts back up where you left off. This is useful for:
- Quick startups – your VM is restored in a running state with all apps open.
- Say you need to reboot your host but don’t want to lose your long running Strategy Wizard session. Save the state and when you restore, OT will continue right where it left off without you having to do anything.
- If you have multiple guest VM’s, saving state allows you to switch between different VM’s and pickup where you left off later. When you save the state, the CPU and Memory resources it consumed on the host are freed up and available for other VM’s or work you need to do on your host.

• I recommend saving the machine state every time you close the VM. If I want to power off the machine, I use the normal Windows shutdown within the guest. If for some reason the guest VM stops responding and hangs, you may need to use the other 2 options. But this is rare. I reboot my Windows VM about once a week and allow it to do its maintenance.
• Take Snapshots when needed.

You'll find the Snapshot function in several places:

• Under the Machine menu you will find Take Snapshot. You can also find this in the VirtualBox Manager.


(Tip-2.png)




Post moved by on 5/21/2018 5:47 PM from General Discussion > Operating System & Software > VirtualBox Quick Tips


Attachments
Attachments Tip-2.png (171KB - 0 downloads)
Top of the page Bottom of the page
JeffBrown
Posted 5/21/2018 5:46 PM (#9092 - in reply to #9090)
Subject: VirtualBox Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
Snapshots can also be found under the VirtualBox Manager

• The show Snapshot button is located on the top right, next to the Machine Tools. If you don’t see it, click the small down arrow next to the Machine Tools and select the Snapshots. This will bring up the Snapshot view for the selected Virtual Machine.

• Snapshots are a term used to represent a moment in time. The state of the machine is captured at a moment in time just like the save state. However, save state is only saved until you start the VM. Snapshots stay around until you remove them. In the image above, you see that there is one snapshot associated with the VM OT-ATM-R. It is called Snapshot 1 (an arbitrary name). You also see the date and time it was taken. If for some reason, I want to go back in time to that point, I can reset the VM using the Restore button that is grayed out in the image. It is important to note that you can take multiple snapshots to represent different points in time.

• Imagine that Windows does an update and the VM refuses to start or suddenly something broke because you installed something or you clicked the wrong link in the browser and now your system has a nasty virus. Simply select the snapshot you want to return to, click restore, and VirtualBox will move you back in time. Of course everything that happened from the date/time of the snapshot is lost, but you now have a functioning VM instead of a broken one.

• Snapshots are also useful for testing. Take a snapshot just prior to making changes to run a test. Execute the test and when done, restore back to the snapshot. Hopefully you saved the results you wanted.

• To take a snapshot, use the Take button in the VirtualBox Manger or use the Machine drop down menu.

• A good practice is to have no more than one or two snapshots on any given VM. Snapshots take up disk space on your host. Behind the scene, VirtualBox creates a special file that represents the differences between the main disk of the VM and the snapshot. This “differencing” file grows as more changes occur inside the VM. To reclaim the space, you simply select the snapshot you don’t want any more and click Delete in the VirtualBox Manager.

• It is best to delete snapshots starting from the oldest but you can deleted snapshots in the middle of a series of snapshots.

• Cloning a point in time snapshot is a powerful feature. Imagine that you had 5 snapshots representing a sequence of changes you made in the VM. You decide that at snapshot 3 reflects a point in time you want to do live trading with but you want to continue your modifications to the current VM. If you Restore back to snapshot 3, you would lose all the work since then. To solve this, you can clone your VM beginning at any snapshot. This creates a new VM that reflects the point in time that you chose. Simply select the snapshot (or current state) from the list of snapshots and click Clone. Follow the prompts.


(Tip-3.png)




Post moved by on 5/21/2018 5:47 PM from General Discussion > Operating System & Software > VirtualBox Quick Tips


Attachments
Attachments Tip-3.png (105KB - 1 downloads)
Top of the page Bottom of the page
SalimHira
Posted 5/21/2018 11:35 PM (#9102 - in reply to #9089)
Subject: Virtualize a Computer & Other Quick Tips



Veteran

Posts: 148
10025
Location:
USA: MD, Columbia
Hi Jeff:

Thanks a bunch for putting together the tutorials, very informative and hands-on.

As I have not yet dived into this territory, to clarify, can I have multiple copies of OT running based on # of OT-installs running concurrently ?

My question as not clear, for example, I believe I can only use specifically only one OT from any given virtual drive at a time ? For example, I may not be able to use concurrently OT2018 Realtime trading stocks from a given virtual drive, while trading at same time realtime futures/forex on another virtual drive from the same laptop / computer ?

I am thinking it cannot be done, but sole reason for having virtual drive(s) is to keep multiple OT's on same computer/laptop for various multiple uses on specific need basis. Am I correct or mistaken? Thanks.


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
JeffBrown
Posted 5/22/2018 9:59 AM (#9103 - in reply to #9102)
Subject: Virtualize a Computer & Other Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
Hi Salim,

Great question. It comes down to 2 things -- license and technology.

On the license side, Nirvana doesn't restrict concurrent execution so you can have up to 5 instances of OT running (at the same time if desired) -- basically 5 installations (or virtual installations). If you go beyond 5, eventually the Nirvana key server will prompt you with each start of OT for your customer # and pw. When that happens, a quick call to Nirvana support (or email) asking to have your license keys reset will make this disappear (assuming you reduce your number to 5). Creating and destroying VM's attached to the network can cause a license lookup under certain conditions and may increment the count at Nirvana too. But Nirvana understands what we are doing and trusts us to honor the license agreement.

On the technology side, since each OT instance is encapsulated within a VM, you can run multiple instances of OT concurrently. So, you could have one installation of OT on your host OS and 2 VM's for a total of 3 -- all running at the same time. I frequently have between 3 and 5 running simultaneously on the same host. The constraint is on your host resources -- memory, CPU, disk space, and disk I/O. So, no OT constraint here either.

You mentioned Real Time Data. I used to use Nirvana's Real Time Data Service. I connected multiple instances of OT to it along with VT -- all running concurrently. There used to be a constraint of one login at a time. A number of years ago when Ed created OTI (Omni Traders International), the team created a Trade Processor as one of the first software products. It used OmniData Real Time. It ran both on your desktop and in the cloud -- some of the first instances were just a desktop installation on the Amazon cloud. This quickly highlighted the need to support multiple logins so the single login restriction was eliminated to facilitate your desktop use and the Trade Processor use. Since then the Trade Processor has evolved along with other services like OmniVest and OmniFunds. But the ability to have multiple simultaneous logins remains.

The last constrained area is the Broker connection. I use GXT (a branding of IB). I frequently run multiple concurrent instances of OT against 2 GXT accounts -- each instance of OT running a different set of strategies. I may also have OmniVest or OmniFunds connected to one of those accounts as well. This means that GXT supports multiple logins. I am also able to use the IB web portal through Garwood and the GXT TradeAway portal while OT is connected. So, with IB, there isn't a constraint here. I can't tell you if there is a constraint with other brokers.

Virtualization has helped me perform both live trading and strategy development/analysis concurrently.

During trading hours, I have my live trading VM's running. During non-trading hours, I have my analysis running -- very useful when Strategy Wizard takes a month to crunch through all my permutations.

Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
SalimHira
Posted 5/22/2018 7:07 PM (#9104 - in reply to #9089)
Subject: Virtualize a Computer & Other Quick Tips



Veteran

Posts: 148
10025
Location:
USA: MD, Columbia
Hi Jeff:

Awesome, your feedback cleared the path to making it imperative of creating and running OT on the virtual drive. It just makes sense and practical now that I understand clearly the powerful potential to work efficiently... and here I was at one time to acquire 2-5 laptops/computers to do what I can now do with one or possibly getaway with 2 computer(s) running multiple OT's on them depending on need basis.

Time now to burn those midnight candles :-).

Thanks again.


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
MarkHolstius
Posted 5/23/2018 8:46 AM (#9105 - in reply to #9103)
Subject: Virtualize a Computer & Other Quick Tips



Veteran

Posts: 204
100100
Location:
USA: IL, Sleepy Hollow
Outstanding Jeff!

Thanks so much for taking the time to put this all together in such a condensed and clear manner.

I'd been hesitant before, but plan on setting up VMs on my laptop now.

Mark

Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
SteveJeary
Posted 5/24/2018 1:48 PM (#9106 - in reply to #9089)
Subject: Virtualize a Computer & Other Quick Tips


Spectator

Posts: 1
0
Location:
UK: , Cardiff
Jeff, thank you.

The work selflessly posted by you, Steve Luerman and one or two others have answered all my questions on this (to me) complex topic.

Steve

Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
SteveLuerman
Posted 6/10/2018 10:08 PM (#9108 - in reply to #9103)
Subject: Virtualize a Computer & Other Quick Tips



Elite

Posts: 407
100100100100
Location:
USA: CO, Boulder
Jeff,
I just saw your posts today (apparently notifications haven't been working for me). You've provided a fantastic how-to guide, plus a lot of very good, well organized advice. Thank you.

It appears to me you're coming from the Mac world. So, this will really help those folks in particular, but it was great for anyone getting into virtualization.

You mentioned Hyper-V. That's what brought me to the forum today, looking for any comments I may have forgotten, or not seen. I'm investigating (well actually plunging into) using the "Client Hyper-V" included in Windows 10 Pro. I hadn't realized (until this weekend) that it is a type-1 hypervisor. I've been running on VMware Workstation Pro, and have seen occasional hangs in OT during ToDo processing. The hangs strike me as possibly symptomatic of a race condition. When it happens, I have to kill OT. I only see it happen in a VM (running in VMWare Workstation - a type-2 hypervisor). So, I'm thinking/hoping that I'll not see that happen with Hyper-V.

Anyway, thanks for your significant work.


(And I just realized, I'll find out if notifications are working for me - I changed to a gmail address for notifications.)

Edited by SteveLuerman 6/10/2018 11:00 PM


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
JeffBrown
Posted 6/11/2018 9:45 AM (#9109 - in reply to #9108)
Subject: Virtualize a Computer & Other Quick Tips



Friend

Posts: 42
25
Location:
WI: ,
Thanks Steve, I appreciate the comments.

I'm actually more of a Linux guy and have very little Mac experience -- I've watched people use it and have a VM under VirtualBox running OS X (a hackintosh experiment) but never made it my primary OS. I've made Linux my main desktop OS since about 2003, starting with transforming a XP laptop into Ubuntu with VirtualBox and a VM of the XP laptop's OS (yes, virtualized the laptop). I still do have a Windows 10 laptop that I use when traveling, but most of the time I have it running 3 VM's of OT, crunching through some long running strategy wizard experiments.

In the past, I've used VMware (server versions and desktop versions), Xen, and KVM. I tried hyper-v (which is based on Xen from what I understand), but never used it for much. I found it less friendly than VMware and VirtualBox -- but that was several years ago. I'm interested in your experience. It seems to me though that you have the hypervisor with Windows 10 in it and then all hyper-v instances are created on top of that Windows 10 instance (creating a nested configuration). I might be wrong with this and Microsoft may have created a way that you seamlessly spawn a sibling VM vs a child VM but allow the Windows 10 instance to control the sibling.

Regardless, I'm going to guess that type 1 vs 2 hypervisor isn't going to fix the occasional hangs of OT. I have had situations where running many strategies or against thousands of symbols (e.g. thinking running 50 strategies against 20,000 symbols over a 10 year period) would be a good idea and having OT give up and spawn errors. I usually find that I'm CPU starved or often Memory starved with the guest OS doing a lot of memory swapping -- grinding OT and the VM to a halt. I've also created bad strategies that used OLang incorrectly causing such inefficiencies that my ToDo runs took days to advance 5%. (Jim helped me see the error of my ways and think differently for OLang).

But your description of occasional hangs doesn't match this, so maybe it is something happening on the host OS and a type-1 would at least move that out of the way. I'm very interested in how it turns out. Do you see it run faster? Is it more stable?

I've also kicked around the idea of putting OT into Docker and seeing if I could run more instances concurrently by eliminating all the extra OS parts that duplicate the base OS, but that's a bit of a science experiment I don't have time for and the ongoing maintenance to the Docker images is more than I want to deal with right now. So that's not at the top of my list.


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
SteveLuerman
Posted 6/11/2018 11:04 AM (#9110 - in reply to #9109)
Subject: Virtualize a Computer & Other Quick Tips



Elite

Posts: 407
100100100100
Location:
USA: CO, Boulder
Hi, Jeff.
I had been thinking of experimenting with the free, bare, CLI version of Hyper-V when I ran across this site (altaro.com), and realized that there was an easier way to run a type-1 hypervisor: https://www.altaro.com/hyper-v/client-hyper-v-windows-10/
I won't reiterate what is stated at that link (and elsewhere at Altaro.com). Comments on other sites also confirm that Client Hyper-V is a type-1 hypervisor.

I agree Client Hyper-V is not quite as user friendly as VMware, but like you I've got 15+ years of experience and comfort with VMware (on Windows). So, it may simply be that they are different (i.e., Hyper-V vs VMware). So, I'll see how it goes, and report back in the future.

I'm pretty sure the hang situation is not a processor or RAM limitation. I like to putter with PC's and have put together a Supermicro 2 socket Xeon system with 16 cores (32 logical processors) and 64GB RAM. Not super fast, but lots of processors for playing with VMs. :^)

And lastly, the hang happens during normal ToDo analysis with 300 symbols and 8 strategies. So, nothing overwhelming. But it ONLY happens on the guest VMs. It has never happened on my host PC.

So, this may all be a grand waste of time, but still interesting and educational.


Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
SteveLuerman
Posted 6/20/2018 10:54 PM (#9111 - in reply to #9110)
Subject: Virtualize a Computer & Other Quick Tips



Elite

Posts: 407
100100100100
Location:
USA: CO, Boulder
Just a quick, short update on my Hyper-V experience:
First, I was on another short 1 week vacation away from OT and Client Hyper-V this past week for family wedding and travel.

But before leaving last week, I had run into one issue with VT and Client Hyper-V. VT requires graphics support. VMware handles that without problems on my current hardware (software emulation? need to research). But Client Hyper-V is excrutiatingly slow in rendering the VT window (i.e., useless) on my hardware. Hyper-V utilizes the graphics hardware for "RemoteFX" - graphics support in VMs. And my graphics hardware (GXForce series) is not in the list of recommended graphics hardware. So, I need to research this further, and possibly replace my graphics card.
Links:
https://social.technet.microsoft.com/wiki/contents/articles/16652.re...
https://blogs.technet.microsoft.com/virtualization/2015/11/23/discre...
https://cloudblogs.microsoft.com/enterprisemobility/2013/11/05/gpu-r...

So for now, I'll maintain at least one VMware VM to run VT.



Thread moved by JimDean on 5/21/2018 5:43 PM from Custom TradeTight Routines > Universal Instructions & Tools > Virtualize a Physical Computer

Top of the page Bottom of the page
Jump to page : 1
Now viewing page 1 [50 msgs/pg]
( E-mail a Link | Printer Version | Search Room )

Owner of site: Jim Dean -- Forum content is confidential, and may not be distributed without written permission.