<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Mar 14, 2018 at 10:56 AM, Frederik Lotter <span dir="ltr"><<a href="mailto:frederik.lotter@netronome.com" target="_blank">frederik.lotter@netronome.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5">On Wed, Mar 14, 2018 at 10:13 AM, Frederik Lotter <span dir="ltr"><<a href="mailto:frederik.lotter@netronome.com" target="_blank">frederik.lotter@netronome.com</a><wbr>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Tue, Mar 13, 2018 at 6:23 PM, Marcin Wojtas <span dir="ltr"><<a href="mailto:mw@semihalf.com" target="_blank">mw@semihalf.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Frederic,<br>
<span class="m_3149656166985352583m_7534793497767103969gmail-"><br>
>>> Please use the size I suggested for the 'reg' property<br>
>><br>
>><br>
>> Sorry I missed that:<br>
>><br>
>> [ 1.463413] PCI: OF: host bridge /cp0/pcie@0xe0000000 ranges:<br>
>> [ 1.463427] PCI: OF: IO 0xeff00000..0xeff0ffff -> 0x00000000<br>
>> [ 1.463435] PCI: OF: MEM 0xc0000000..0xdfffffff -> 0xc0000000<br>
>> [ 1.463442] PCI: OF: MEM 0x800000000..0x8ffffffff -> 0x800000000<br>
>> [ 1.463481] pci-host-generic e0000000.pcie: ECAM at [mem<br>
>> 0xe0000000-0xefefffff] for [bus 00-fe]<br>
>> [ 1.463525] pci-host-generic e0000000.pcie: PCI host bridge to bus<br>
>> 0000:00<br>
>> [ 1.463531] pci_bus 0000:00: root bus resource [bus 00-fe]<br>
>> [ 1.463536] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]<br>
>> [ 1.463541] pci_bus 0000:00: root bus resource [mem<br>
>> 0xc0000000-0xdfffffff]<br>
>> [ 1.463547] pci_bus 0000:00: root bus resource [mem<br>
>> 0x800000000-0x8ffffffff]<br>
>><br>
>><br>
>> So I assume this works, and I am super grateful. I will test it tomorrow<br>
>> with our Smart NIC.<br>
>><br>
<br>
</span>Please also check pure branch (unmodified DT) as requested in my<br>
previous email . According to the bootlog the stall was observed after<br>
pcie driver init and without latest patch I mentioned. So I'd like to<br>
make sure that we're on the same side here.<br></blockquote><div> </div></span><div>I attached two logs files. The only difference is the one also uses an initrd.</div><div><br></div><div>I now have a pcie card in - and it looks much better.</div><div><br></div><div>I am starting to feel like this whole exercise is going to discover something unrelated and stupid I have done, so I apologize in advance. Hopefully the journey will help someone else as well.</div><div><br></div><div>Any idea what could be failing? I seem to be some issue mounting the root filesystem
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">(or at the same time). In the initrd case it seems to complain about mmcblk0 (the onboard flash which I do not use).</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I attach my grub file just for in case - i am new to grub (Note I hacked 4 entries towards the end).</span></div></div></div></div></blockquote><div><br></div></div></div><div>Here are 3 more logs, I will not send more - but I thought perhaps this could complete the picture.</div><div><br></div><div>1. One without a PCIe card - successful PCIe init</div><div><br></div><div>2. One with a card, but with the PCIe probe causing the stall (this is not often seen)</div><div><br></div><div>3. The other stall, after successful PCIe init</div></div></div></div></blockquote><div><br></div><div>Hi Marcin, </div><div><br></div><div>I am sure you are quite busy, and I really appreciate all the help I got from you.</div><div><br></div><div>Please will you have a look at the last two emails (and the attachments) I've sent you, once you have time again. If there is anything I can do that will help you, just let me know. I really need to get this working reliably for us to proceed with the EFI boot route, and since we really need to support generic netboot/ISO installs and images for CentOS and Ubuntu, I think this must be the best way forward.</div><div><br></div><div>Regards,</div><div>Fred</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="m_3149656166985352583h5"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="m_3149656166985352583m_7534793497767103969gmail-"><br>
><br>
> This doesn't tell you all that much, to be honest. But at least the<br>
> numbers look sane now, and appear to match the UEFI configuration.<br>
><br>
>> However, we are building a product that obviously requires long term<br>
>> maintenance, so may I please get your input on a strategy with this?<br>
>><br>
>> If we decide to stick with this driver, would it be easy for things to<br>
>> become disjointed?<br>
>><br>
>> The hope with going the EFI route is that we could boot "generic" Ubuntu and<br>
>> CentOS installs, so I guess as long as we keep the DT and the EFKII snapshot<br>
>> in sync on our side, the risk is low.<br>
>><br>
><br>
> I'm afraid you are getting caught in the middle of a philosophical<br>
> debate here: many engineers that are involved with the Marvell support<br>
> in Linux feel that a device tree is not something that should be<br>
> supported long term, and needs to be bundled with the OS. Over the<br>
> last couple of kernel releases, the Marvell 8040 support was changed<br>
> in a non-backward compatible manner numerous times.<br>
><br>
<br>
</span>I think current DT should work with everything >= v4.12. So far<br>
multiple users were able to install debian with recent fixes, I<br>
suggest first making sure, what possibly can happen that your setup<br>
behaves differently. Switching to a8k-ecam-pcie driver is a nice idea,<br>
but I'm not sure the distros using DT have it.<br>
<span class="m_3149656166985352583m_7534793497767103969gmail-"><br>
> This conflicts badly with the idea that the firmware provides the<br>
> hardware description (using DT or ACPI), and that the contract with<br>
> the OS is kept by both sides for longer than a single release.<br>
><br>
> So I cannot really answer that question, unfortunately. If you don't<br>
> intend to use the onboard network controller, you could go the ACPI<br>
> route, I guess.<br>
<br>
</span>FYI. on-board network ACPI support is being upstreamed to the Centos.<br>
<span class="m_3149656166985352583m_7534793497767103969gmail-"><br>
><br>
> Another problem is that none of this UEFI/ACPI support is upstream in<br>
> the Tianocore project, and trying random trees left and right doesn't<br>
> really help when assessing whether a platform is suitable as a long<br>
> term investment.<br>
><br>
<br>
</span>There's only single branch recommended in the MacchiatoBin wiki, I<br>
wouldn't call it 'random'. Entire branch is supposed to land<br>
eventually in the Tianocore and become the only support. Before end of<br>
year ~50 patches got there, still some bits are missing, but I think<br>
we're not that far from desired point. I really want to push it but<br>
still it requires time I'm personally short of, so I'll appreciate<br>
understanding.<br>
<br>
Thanks,<br>
Marcin<br>
<div class="m_3149656166985352583m_7534793497767103969gmail-HOEnZb"><div class="m_3149656166985352583m_7534793497767103969gmail-h5"><br>
><br>
><br>
>> For example, using the same DT with uboot, it fails:<br>
>><br>
>> [ 0.294942] sysfs: cannot create duplicate filename<br>
>> '/bus/platform/devices/e000000<wbr>0.pcie'<br>
>> [ 0.294950] CPU: 2 PID: 1 Comm: swapper/0 Not tainted<br>
>> 4.16.0-rc5-mbcin-netronome-2-d<wbr>irty #2<br>
>> [ 0.294952] Hardware name: Marvell 8040 MACHIATOBin (DT)<br>
>> [ 0.294955] Call trace:<br>
>> [ 0.294967] dump_backtrace+0x0/0x150<br>
>> [ 0.294970] show_stack+0x14/0x20<br>
>> [ 0.294976] dump_stack+0x98/0xbc<br>
>> [ 0.294980] sysfs_warn_dup+0x60/0x78<br>
>> [ 0.294983] sysfs_do_create_link_sd.isra.0<wbr>+0xd8/0xe0<br>
>> [ 0.294986] sysfs_create_link+0x20/0x40<br>
>> [ 0.294990] bus_add_device+0x88/0x148<br>
>> [ 0.294993] device_add+0x394/0x568<br>
>> [ 0.294997] of_device_add+0x5c/0x70<br>
>> [ 0.295000] of_platform_device_create_pdat<wbr>a+0x80/0xd0<br>
>> [ 0.295003] of_platform_bus_create+0xdc/0x<wbr>300<br>
>> [ 0.295006] of_platform_bus_create+0x11c/0<wbr>x300<br>
>> [ 0.295008] of_platform_populate+0x4c/0xb0<br>
>> [ 0.295014] of_platform_default_populate_i<wbr>nit+0xa4/0xc0<br>
>> [ 0.295017] do_one_initcall+0x38/0x120<br>
>> [ 0.295020] kernel_init_freeable+0x134/0x1<wbr>d4<br>
>> [ 0.295025] kernel_init+0x10/0x100<br>
>> [ 0.295028] ret_from_fork+0x10/0x18<br>
>><br>
>> So I think this confirms that the pcie setup is different between EDKII and<br>
>> uboot (unless I am doing something stupid here).<br>
>><br>
><br>
> It looks like you have two copies of the pcie node here, no?<br>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>