[Macchiato] EDKII grub boot fails with PCIe init

Fri Mar 23 07:07:26 GMT 2018

On Fri, Mar 16, 2018 at 5:16 PM, Marcin Wojtas <mw at semihalf.com> wrote:

> Hi Frederik,
>
> 2018-03-16 14:04 GMT+01:00 Frederik Lotter <frederik.lotter at netronome.com
> >:
> >
> > On Wed, Mar 14, 2018 at 10:56 AM, Frederik Lotter <
> frederik.lotter at netronome.com> wrote:
> >>
> >> On Wed, Mar 14, 2018 at 10:13 AM, Frederik Lotter <
> frederik.lotter at netronome.com> wrote:
> >>>
> >>> On Tue, Mar 13, 2018 at 6:23 PM, Marcin Wojtas <mw at semihalf.com>
> wrote:
> >>>>
> >>>> Frederic,
> >>>>
> >>>> >>> Please use the size I suggested for the 'reg' property
> >>>> >>
> >>>> >>
> >>>> >> Sorry I missed that:
> >>>> >>
> >>>> >> [    1.463413] PCI: OF: host bridge /cp0/pcie at 0xe0000000 ranges:
> >>>> >> [    1.463427] PCI: OF:    IO 0xeff00000..0xeff0ffff -> 0x00000000
> >>>> >> [    1.463435] PCI: OF:   MEM 0xc0000000..0xdfffffff -> 0xc0000000
> >>>> >> [    1.463442] PCI: OF:   MEM 0x800000000..0x8ffffffff ->
> 0x800000000
> >>>> >> [    1.463481] pci-host-generic e0000000.pcie: ECAM at [mem
> >>>> >> 0xe0000000-0xefefffff] for [bus 00-fe]
> >>>> >> [    1.463525] pci-host-generic e0000000.pcie: PCI host bridge to
> bus
> >>>> >> 0000:00
> >>>> >> [    1.463531] pci_bus 0000:00: root bus resource [bus 00-fe]
> >>>> >> [    1.463536] pci_bus 0000:00: root bus resource [io
> 0x0000-0xffff]
> >>>> >> [    1.463541] pci_bus 0000:00: root bus resource [mem
> >>>> >> 0xc0000000-0xdfffffff]
> >>>> >> [    1.463547] pci_bus 0000:00: root bus resource [mem
> >>>> >> 0x800000000-0x8ffffffff]
> >>>> >>
> >>>> >>
> >>>> >> So I assume this works, and I am super grateful. I will test it
> tomorrow
> >>>> >> with our Smart NIC.
> >>>> >>
> >>>>
> >>>> Please also check pure branch (unmodified DT) as requested in my
> >>>> previous email . According to the bootlog the stall was observed after
> >>>> pcie driver init and without latest patch I mentioned. So I'd like to
> >>>> make sure that we're on the same side here.
> >>>
> >>>
> >>> I attached two logs files. The only difference is the one also uses an
> initrd.
> >>>
> >>> I now have a pcie card in - and it looks much better.
> >>>
> >>> I am starting to feel like this whole exercise is going to discover
> something unrelated and stupid I have done, so I apologize in advance.
> Hopefully the journey will help someone else as well.
> >>>
> >>> Any idea what could be failing? I seem to be some issue mounting the
> root filesystem  (or at the same time). In the initrd case it seems to
> complain about mmcblk0 (the onboard flash which I do not use).
> >>>
> >>> I attach my grub file just for in case - i am new to grub (Note I
> hacked 4 entries towards the end).
> >>
> >>
> >> Here are 3 more logs, I will not send more - but I thought perhaps this
> could complete the picture.
> >>
> >> 1. One without a PCIe card - successful PCIe init
> >>
> >> 2. One with a card, but with the PCIe probe causing the stall (this is
> not often seen)
> >>
> >> 3. The other stall, after successful PCIe init
> >
> >
> > Hi Marcin,
> >
> > I am sure you are quite busy, and I really appreciate all the help I got
> from you.
> >
> > Please will you have a look at the last two emails (and the attachments)
> I've sent you, once you have time again. If there is anything I can do that
> will help you, just let me know. I really need to get this working reliably
> for us to proceed with the EFI boot route, and since we really need to
> support generic netboot/ISO installs and images for CentOS and Ubuntu, I
> think this must be the best way forward.
> >
>
> I took a look at your logs and it all looks a bit strange. Is it pure
> v4.16-rc5? If yes, can you avoid grub and boot directly from shell?
>

Could you give me a hint how I do this? I am very new to EDK/grub?

>
> Can you share your grub? I'd like to test it in my setup.
>

Please find the content of my EFI partition and /boot/grub attached. I will
really appreciate if you could give it a go.

I got this grub through apt-get for Ubuntu.

apt-get install grub-efi-arm64

Info below:

root at localhost:/boot# apt-get install grub-efi-arm64
Reading package lists... Done
Building dependency tree
Reading state information... Done
grub-efi-arm64 is already the newest version (2.02~beta2-36ubuntu3.17).
0 upgraded, 0 newly installed, 0 to remove and 48 not upgraded.
root at localhost:/boot# uname -a
Linux localhost.localdomain 4.16.0-rc5-mbcin-netronome-2-dirty #2 SMP
PREEMPT Mon Mar 12 14:40:25 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
root at localhost:/boot# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

>
> Can you please download:
> https://d-i.debian.org/daily-images/arm64/20180314-02:10/netboot/mini.iso
> burn on stick and install? It's very easy and I use it in my GPU setup
>

OK I will try this but first I want to redo my environment from scratch so
make sure I have not messed up anything.

> on MacchiatoBin.
>
> Thanks,
> Marcin
>
> >
> >>
> >>>
> >>>>
> >>>> >
> >>>> > This doesn't tell you all that much, to be honest. But at least the
> >>>> > numbers look sane now, and appear to match the UEFI configuration.
> >>>> >
> >>>> >> However, we are building a product that obviously requires long
> term
> >>>> >> maintenance, so may I please get your input on a strategy with
> this?
> >>>> >>
> >>>> >> If we decide to stick with this driver, would it be easy for
> things to
> >>>> >> become disjointed?
> >>>> >>
> >>>> >> The hope with going the EFI route is that we could boot "generic"
> Ubuntu and
> >>>> >> CentOS installs, so I guess as long as we keep the DT and the
> EFKII snapshot
> >>>> >> in sync on our side, the risk is low.
> >>>> >>
> >>>> >
> >>>> > I'm afraid you are getting caught in the middle of a philosophical
> >>>> > debate here: many engineers that are involved with the Marvell
> support
> >>>> > in Linux feel that a device tree is not something that should be
> >>>> > supported long term, and needs to be bundled with the OS. Over the
> >>>> > last couple of kernel releases, the Marvell 8040 support was changed
> >>>> > in a non-backward compatible manner numerous times.
> >>>> >
> >>>>
> >>>> I think current DT should work with everything >= v4.12. So far
> >>>> multiple users were able to install debian with recent fixes, I
> >>>> suggest first making sure, what possibly can happen that your setup
> >>>> behaves differently. Switching to a8k-ecam-pcie driver is a nice idea,
> >>>> but I'm not sure the distros using DT have it.
> >>>>
> >>>> > This conflicts badly with the idea that the firmware provides the
> >>>> > hardware description (using DT or ACPI), and that the contract with
> >>>> > the OS is kept by both sides for longer than a single release.
> >>>> >
> >>>> > So I cannot really answer that question, unfortunately. If you don't
> >>>> > intend to use the onboard network controller, you could go the ACPI
> >>>> > route, I guess.
> >>>>
> >>>> FYI. on-board network ACPI support is being upstreamed to the Centos.
> >>>>
> >>>> >
> >>>> > Another problem is that none of this UEFI/ACPI support is upstream
> in
> >>>> > the Tianocore project, and trying random trees left and right
> doesn't
> >>>> > really help when assessing whether a platform is suitable as a long
> >>>> > term investment.
> >>>> >
> >>>>
> >>>> There's only single branch recommended in the MacchiatoBin wiki, I
> >>>> wouldn't call it 'random'. Entire branch is supposed to land
> >>>> eventually in the Tianocore and become the only support. Before end of
> >>>> year ~50 patches got there, still some bits are missing, but I think
> >>>> we're not that far from desired point. I really want to push it but
> >>>> still it requires time I'm personally short of, so I'll appreciate
> >>>> understanding.
> >>>>
> >>>> Thanks,
> >>>> Marcin
> >>>>
> >>>> >
> >>>> >
> >>>> >> For example, using the same DT with uboot, it fails:
> >>>> >>
> >>>> >> [    0.294942] sysfs: cannot create duplicate filename
> >>>> >> '/bus/platform/devices/e0000000.pcie'
> >>>> >> [    0.294950] CPU: 2 PID: 1 Comm: swapper/0 Not tainted
> >>>> >> 4.16.0-rc5-mbcin-netronome-2-dirty #2
> >>>> >> [    0.294952] Hardware name: Marvell 8040 MACHIATOBin (DT)
> >>>> >> [    0.294955] Call trace:
> >>>> >> [    0.294967]  dump_backtrace+0x0/0x150
> >>>> >> [    0.294970]  show_stack+0x14/0x20
> >>>> >> [    0.294976]  dump_stack+0x98/0xbc
> >>>> >> [    0.294980]  sysfs_warn_dup+0x60/0x78
> >>>> >> [    0.294983]  sysfs_do_create_link_sd.isra.0+0xd8/0xe0
> >>>> >> [    0.294986]  sysfs_create_link+0x20/0x40
> >>>> >> [    0.294990]  bus_add_device+0x88/0x148
> >>>> >> [    0.294993]  device_add+0x394/0x568
> >>>> >> [    0.294997]  of_device_add+0x5c/0x70
> >>>> >> [    0.295000]  of_platform_device_create_pdata+0x80/0xd0
> >>>> >> [    0.295003]  of_platform_bus_create+0xdc/0x300
> >>>> >> [    0.295006]  of_platform_bus_create+0x11c/0x300
> >>>> >> [    0.295008]  of_platform_populate+0x4c/0xb0
> >>>> >> [    0.295014]  of_platform_default_populate_init+0xa4/0xc0
> >>>> >> [    0.295017]  do_one_initcall+0x38/0x120
> >>>> >> [    0.295020]  kernel_init_freeable+0x134/0x1d4
> >>>> >> [    0.295025]  kernel_init+0x10/0x100
> >>>> >> [    0.295028]  ret_from_fork+0x10/0x18
> >>>> >>
> >>>> >> So I think this confirms that the pcie setup is different between
> EDKII and
> >>>> >> uboot (unless I am doing something stupid here).
> >>>> >>
> >>>> >
> >>>> > It looks like you have two copies of the pcie node here, no?
> >>>
> >>>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.einval.com/pipermail/macchiato/attachments/20180323/0ddf0b92/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: boot-grub.tar.gz
Type: application/x-gzip
Size: 3004951 bytes
Desc: not available
URL: <http://lists.einval.com/pipermail/macchiato/attachments/20180323/0ddf0b92/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: EFI.tar.gz
Type: application/x-gzip
Size: 68232 bytes
Desc: not available
URL: <http://lists.einval.com/pipermail/macchiato/attachments/20180323/0ddf0b92/attachment-0003.bin>