J.O. Aho wrote:

> Aragorn wrote:
>
>> Either way, the author of the article was attempting to hide the device
>> from the hypervisor/dom0 and thus had no need for any IOMMU, in software
>> or otherwise. I'm also not so sure on how his experiment - even with the
>> effect of the Windows Remote Desktop Protocol or the reputed poor
>> stability of Windows itself - could possibly lock up his machine solidly,
>> unless there is a serious bug in the hypervisor code. But then again,
>> hardware-based virtualization is a different beast from
>> paravirtualization.
>>
>> Normally, when hardware-based virtualization is used, the hypervisor
>> and /dom0/ run in the CPU's root mode - which has its own four privilege
>> rings - while /domU/ virtual machines run in the CPU's non-root mode -
>> equally with four privilege levels. So technically, if the machine
>> locked up solid, then this means that something has gone wrong in the
>> root mode of the CPU, and thus in the hypervisor code - unless of course
>> we're talking of a bug/flaw in Intel's Vanderpool technology.
>>
>> Still, it was an interesting read, and certainly food for thought, thank
>> you. :-)
>
> I'm following your progress with interest here, as some of the ideas I
> have had, you are now trying to do, mainly this with graphics access for
> the domU.
> What I lack at the moment is a machine powerful enough to try Xen on.
>
> All Xen experience I have is the little I did experimented at work, but
> that was just to run a number of webserver which don't require graphics
> output as a virtual media player.

I currently haven't gotten very far yet. In fact, I still am in that
chroot'ed environment and I still have to configure and build a kernel, but
unfortunately lots of other stuff has kept me busy these last few days.

Now that I know you're interested in my "project" I will of course keep you
posted and fill you on on what I have (or have not) come to accomplish. ;-)

P.S.: It would seem that I was wrong in the post from which you quoted...
As it turns out, the newer and more powerful Intel-based machines do indeed
have an on-board IOMMU, but as is also the case for the regular memory
controller, it's on the motherboard, contrary to AMD-based platforms, where
the memory controller and IOMMU are part of the processor chip itself.

Intel did however announce that their future line of processor chips will
also have an on-chip memory controller, which means that they too are
realizing the benefits of a ccNUMA architecture for the x86 platform. As a
sidenote to this, the upcoming 2.6.25 kernel will no longer even support a
flat memory model as the sparse memory model has overtaken both the
contiguous and flat models and seems to work equally well for NUMA and
non-NUMA machines. ;-)

--
Aragorn
(registered GNU/Linux user #223157)

Aragorn wrote:

> I currently haven't gotten very far yet. In fact, I still am in that
> chroot'ed environment and I still have to configure and build a kernel, but
> unfortunately lots of other stuff has kept me busy these last few days.

That is life, always something coming in between.
As I noticed it goes quite painlessly to setup a basic environment on a pre
compiled distro, but then you loose most of the customizing. I think it took
me like a hour to setup Xen on CentOS, the main problem I had was how to
access the RPMs over http, until I figured out to use the external ip-number
of the host system and then everything went fine.
Something I did a bit later on was to install the Xen on it's own slices
instead of a image file, which seems to be the default setup, but I have to
say I don't like the image files, they tend to be slow and require more CPU
when doing file I/O.


> Now that I know you're interested in my "project" I will of course keep you
> posted and fill you on on what I have (or have not) come to accomplish. ;-)

I look forward for your posts and I will keep on trying to make some inputs,
which may sometimes not be that much worth.


> P.S.: It would seem that I was wrong in the post from which you quoted...
> As it turns out, the newer and more powerful Intel-based machines do indeed
> have an on-board IOMMU, but as is also the case for the regular memory
> controller, it's on the motherboard, contrary to AMD-based platforms, where
> the memory controller and IOMMU are part of the processor chip itself.
>
> Intel did however announce that their future line of processor chips will
> also have an on-chip memory controller, which means that they too are
> realizing the benefits of a ccNUMA architecture for the x86 platform. As a
> sidenote to this, the upcoming 2.6.25 kernel will no longer even support a
> flat memory model as the sparse memory model has overtaken both the
> contiguous and flat models and seems to work equally well for NUMA and
> non-NUMA machines. ;-)

Ok, thats good to know, recently went to kernel 2.6.24 on all my machines
except the Sparc, which I haven't turned on for 6 month, and starting to think
of getting rid of it, it's too slow for me and too noisy even if I have
switched to more quite fans on the machine.

You know if there is plans of merge Xen into the kernel source or will it keep
on being a separated packages?


--

//Aho

J.O. Aho wrote:

> Something I did a bit later on was to install the Xen on it's own slices
> instead of a image file, which seems to be the default setup, but I have
> to say I don't like the image files, they tend to be slow and require more
> CPU when doing file I/O.

Indeed, file-backed filesystems are always slower than real "slices". In my
case, I have opted for all regular partitions for the XenLinux host system
(i.o.w.,the /dom0/ machine) with the exception of */usr/src* and
*/usr/portage,* which I have put as logical volumes inside a volume group.

The /domU/ virtual machines all have their filesystems on LVM2 slices, with
each /domU/ machine having its own dedicated volume group. */usr/src* and
*/usr/portage* will be shared by /dom0/ to /domU/ via NFS, and */tmp* and
*/var/tmp* are /tmpfs/ filesystems.

I intend to set up the /domU/ machines via the same /chroot/ technique as
used for installing Gentoo. I'm contemplating the use of a stage4 tarball
to populate the /domU/ machines, but I still have to figure that one
out. ;-)

>> Now that I know you're interested in my "project" I will of course keep
>> you posted and fill you on on what I have (or have not) come to
>> accomplish. ;-)
>
> I look forward for your posts and I will keep on trying to make some
> inputs, which may sometimes not be that much worth.

Still, they will be appreciated. ;-)

>> P.S.: It would seem that I was wrong in the post from which you quoted...
>> As it turns out, the newer and more powerful Intel-based machines do
>> indeed have an on-board IOMMU, but as is also the case for the regular
>> memory controller, it's on the motherboard, contrary to AMD-based
>> platforms, where the memory controller and IOMMU are part of the
>> processor chip itself.
>>
>> Intel did however announce that their future line of processor chips will
>> also have an on-chip memory controller, which means that they too are
>> realizing the benefits of a ccNUMA architecture for the x86 platform. As
>> a sidenote to this, the upcoming 2.6.25 kernel will no longer even
>> support a flat memory model as the sparse memory model has overtaken both
>> the contiguous and flat models and seems to work equally well for NUMA
>> and non-NUMA machines. ;-)
>
> Ok, thats good to know, recently went to kernel 2.6.24 on all my machines
> except the Sparc, which I haven't turned on for 6 month, and starting to
> think of getting rid of it, it's too slow for me and too noisy even if I
> have switched to more quite fans on the machine.
>
> You know if there is plans of merge Xen into the kernel source or will it
> keep on being a separated packages?

Well, I have checked out the vanilla 2.6.24 when I was looking to build my
bootable kernel for that machine, but I've noticed a few things.
Apparently, Xen has already been merged into the mainline kernels as of
2.6.23, *but* there are a few caveats...:

(1) Xen support in the mainline kernel tree is _only_ for use of the kernel
as a Xen *guest,* _not_ as a Xen host.

(2) It is not easy to patch the official kernel sources for use with Xen, so
the current implementation - which was and still is contributed by one of
the official Xen developers - is still quite limited, compared to the
somewhat older 2.6.18 kernels that come with the Xen package itself.
_Note:_ As of a few days ago, Xen 3.2.0 is in Portage with a Ubuntu-patched
2.6.22 kernel, and the Xen 3.1.3 package in Portage uses a RedHat-patched
2.6.20 kernel.

(3) Official mainline Linux 2.6.24.2 - i.e. the latest stable kernel from
kernel.org - does _not_ support use as a 64-bit Xen guest, and SMP is
broken. So if you use a vanilla kernel 2.6.23(.x) or 2.6.24(.x), then the
only way to use it as a Xen guest machine is if the kernel is uniprocessor
and 32-bit-only. I do believe PAE whould work, but I'm not sure.

So the bottom line is that if you want a 64-bit guest or an SMP guest, you
should use the kernel from the Gentoo Xen kernel packages or from the
official xen.org packages.

The good news however is that work is on the way to bring Xen guest support
up to par with the rest. The merging of a lot of files from the /i386/
and /x86_64/ branches of the development 2.6.25 source tree into a largely
unified /x86/ tree has brought several Xen and other paravirtualization
options (such as lguest) to x86_64 already. The bad news is that this
kernel tree is still in /-rc1/ stage as we speak, and it may still take
quite some time before 2.6.25 is officially released. ;-)

Although I have missed a few in the past, I normally read through the entire
Changelog with each new release candidate and each new official release. I
do of course skip that which I find less interesting - such as
network-related stuff or other architectures such as IA64 or PPC - but
still, the 2.6.25-rc1 Changelog was over 140.000 lines of plain text.

It took me several hours to read just the stuff I was interested in, and a
lot in this particular Changelog had to do with /x86,/ both regarding
patches and improvements and regarding the merging of 32-bit and 64-bit
into one unified branch.

With that amount of text, you really do have to switch on a mental filter
and just skip stuff that you find less relevant. What I personally find
most interesting is the /x86/ stuff, the Xen/KVM/lguest stuff, NUMA stuff,
SCSI/SAS stuff, XFS, framebuffer devices, and generic memory management,
security and process handling stuff. I don't quite follow the other
hardware platforms, and PATA and networking are less interesting to me.
Still, some of it grabs my attention sometimes, though. ;-)

--
Aragorn
(registered GNU/Linux user #223157)

Aragorn wrote:

> J.O. Aho wrote:
>
>> I look forward for your posts and I will keep on trying to make some
>> inputs, which may sometimes not be that much worth.

A short follow-up... I've decided to post a request for advice on the topic
of hiding the Radeon PCI adapter - currently the primary video card and the
one I intend to use in /dom0/ only - from /domU/ and/or changing the
primary videocard via a kernel parameter to /comp.os.linux.hardware./ I'll
let you know if anything comes up there. ;-)

--
Aragorn
(registered GNU/Linux user #223157)