<div dir="ltr">Hi Marcin,<div><br></div><div>I am attaching text files, not sure how it works with the list - tell me if there is another way.<br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 13, 2018 at 11:27 AM, Marcin Wojtas <span dir="ltr"><<a href="mailto:mw@semihalf.com" target="_blank">mw@semihalf.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">2018-03-13 8:54 GMT+01:00 Frederik Lotter <<a href="mailto:frederik.lotter@netronome.com">frederik.lotter@netronome.com</a><wbr>>:<br>
> On Tue, Mar 13, 2018 at 8:48 AM, Marcin Wojtas <<a href="mailto:mw@semihalf.com">mw@semihalf.com</a>> wrote:<br>
>><br>
>> 2018-03-13 6:49 GMT+01:00 Frederik Lotter <<a href="mailto:frederik.lotter@netronome.com">frederik.lotter@netronome.com</a><wbr>>:<br>
>> > On 12 Mar 2018 6:45 PM, "Marcin Wojtas" <<a href="mailto:mw@semihalf.com">mw@semihalf.com</a>> wrote:<br>
>> ><br>
>> > Yes, it's the latest - I've just pushed a fix for OS variables write,<br>
>> > so you can fetch. I have a couple of questions:<br>
>> ><br>
>> > 0. What version of the board are you using (is the PCIE-reset fix<br>
>> > there)?<br>
>> ><br>
>> ><br>
>> > We have v1.3 revision boards.<br>
>> ><br>
>> ><br>
>> > 1. What kind of pcie device are you using?<br>
>> ><br>
>> ><br>
>> > We have Smart NIC cards, but this board has nothing plugged in.<br>
>> ><br>
>> ><br>
>> > 2. Does the boot from u-boot always succeed with the pcie device?<br>
>> ><br>
>> ><br>
>> > Yes. It appears 100% reliable in comparison with EFI boot.<br>
>> ><br>
>> > So my conclusion is either the EDK Phy setup is different, or the DT<br>
>> > data is<br>
>> > somehow different.<br>
>><br>
>> Phy setup should be aligned, soon it will be common fortunately. DT is<br>
>> a bit different as it uses 512MB MMIO32 region, but his should not<br>
>> have any impact. Overall this is strange, I've booted kernel hundreds<br>
>> of times without issues. However I usually had at least e1000 NIC<br>
>> plugged in my setup (now also x4 GPU card on the second board). So I<br>
>> have a couple of requests, so that I can better understand the issue:<br>
>><br>
>> 1. Can you provide me with full boot log (beginning from first lines<br>
>> after power-on/reset) from both u-boot and uefi?<br></div></div></blockquote><div><br></div><div>Attached:</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">
>><br>
>> 2. Can you plug the smart nic and try efi boot to see if the problem<br>
>> persists?<br></div></div></blockquote><div><br></div><div>Will try tomorrow - I am working from home and cannot locate anything to plug in.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">
>><br>
>> 3. Can you try cross-checking the DT<br>
>> - take armada-8040-mcbin.dtb from Platforms/Marvell/Armada and boot<br>
>> your linux from u-boot?<br>
>> - take the DT from u-boot and substitute above file in uefi, rebuild<br>
>> your image and boot?<br>
><br>
><br>
> I will do the above, but I have replaced the armada-8040-mcbin.dtb in the<br>
> edk2-platforms with the files generated by the kernel build.<br>
><br>
> DTS: ~/work/edk2-open-platform/<wbr>Platforms/Marvell/Armada/<wbr>armada-8040-db.dts<br>
> <-- linux/arch/arm64/boot/dts/<wbr>marvell/.armada-8040-mcbin.<wbr>dtb.dts.tmp<br>
> DTB: ~/work/edk2-open-platform/<wbr>Platforms/Marvell/Armada/<wbr>armada-8040-db.dtb<br>
> <-- linux/arch/arm64/boot/dts/<wbr>marvell/.armada-8040-mcbin.dtb<br>
><br>
> I did md5sum on both sets and they match on both sides.<br>
><br>
> Let me know if you still want me to try something here.<br>
><br>
<br>
</div></div>Yes, check it :)<br>
<br>
Also please boot unmodified marvell-armada-wip with updated DT:<br>
Lastest commit is (b316449f24fda)<br></blockquote><div><br></div><div>I updated to the latest commit for edk2-platforms. </div><div><br></div><div>Note, there appear to be new commits to edk2 today which breaks at least my ARM64 build.</div><div><br></div><div>So the original edk2-platforms DT succeeds under uboot, fails under EFI.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
<br>
>><br>
>> Thanks,<br>
>> Marcin<br>
>><br>
>> ><br>
>> ><br>
>> > Thanks,<br>
>> > Marcin<br>
>> ><br>
>> ><br>
>> > 2018-03-12 17:31 GMT+01:00 Frederik Lotter<br>
>> > <<a href="mailto:frederik.lotter@netronome.com">frederik.lotter@netronome.com</a><wbr>>:<br>
>> >> 150e1b404d2e3b49574ad57e485827<wbr>be12270ab9<br>
>> >><br>
>> >> I believe this is HEAD, committed on 6th of March?<br>
>> >><br>
>> >> On Mon, Mar 12, 2018 at 6:23 PM, Marcin Wojtas <<a href="mailto:mw@semihalf.com">mw@semihalf.com</a>> wrote:<br>
>> >>><br>
>> >>> Hi Frederik,<br>
>> >>><br>
>> >>> Which commit ID of marvell-armada-wip you're building your binary<br>
>> >>> from?<br>
>> >>><br>
>> >>> Thanks,<br>
>> >>> Marcin<br>
>> >>><br>
>> >>> 2018-03-12 17:22 GMT+01:00 Ard Biesheuvel <<a href="mailto:ard.biesheuvel@linaro.org">ard.biesheuvel@linaro.org</a>>:<br>
>> >>> > On 12 March 2018 at 16:15, Frederik Lotter<br>
>> >>> > <<a href="mailto:frederik.lotter@netronome.com">frederik.lotter@netronome.com</a><wbr>> wrote:<br>
>> >>> >> Hi,<br>
>> >>> >> I am getting CPU stall warnings when booting up using the EFI<br>
>> >>> >> route. I<br>
>> >>> >> suspect the PCIe interface, as the stall warning sometimes contain<br>
>> >>> >> the<br>
>> >>> >> probe<br>
>> >>> >> function. Other times is seems to get further than PCIe init, but<br>
>> >>> >> still<br>
>> >>> >> stall interrupt handling.<br>
>> >>> >> Here are some facts around my observation:<br>
>> >>> >><br>
>> >>> >> I have two sdcards for my Machiattobin board. They have identical<br>
>> >>> >> kernels<br>
>> >>> >> (4.16 rc5) with Ubuntu 16.04 rootfs. The one sdcard uses a uboot,<br>
>> >>> >> DT<br>
>> >>> >> and<br>
>> >>> >> kernel boot. The second sdcard has EDKII, grub kernel boot. The<br>
>> >>> >> EDKII<br>
>> >>> >> build<br>
>> >>> >> includes the device tree DTB (and DTS which I believe is unused)<br>
>> >>> >> from<br>
>> >>> >> the<br>
>> >>> >> one used on the uboot sdcard.<br>
>> >>> >><br>
>> >>> >> EFI stub: Booting Linux Kernel...<br>
>> >>> >> EFI stub: Using DTB from configuration table<br>
>> >>> >> EFI stub: Exiting boot services and installing virtual address<br>
>> >>> >> map...<br>
>> >>> >> [ 0.000000] Booting Linux on physical CPU 0x0000000000<br>
>> >>> >> [0x410fd081]<br>
>> >>> >> [ 0.000000] Linux version 4.16.0-rc5-mbcin-netronome-2-<wbr>dirty<br>
>> >>> >> (root@mcb1-cpt) (gcc version 5.4.0 20160609 (Ubuntu/Linaro<br>
>> >>> >> 5.4.0-6ubuntu1~16.04.9)) #2 SMP PREEMPT Mon Mar 12 14:40:25 UTC<br>
>> >>> >> 2018<br>
>> >>> >> [ 0.000000] Machine model: Marvell 8040 MACHIATOBin<br>
>> >>> >> [ 0.000000] efi: Getting EFI parameters from FDT:<br>
>> >>> >> [ 0.000000] efi: EFI v2.70 by EDK II<br>
>> >>> >> [ 0.000000] efi: SMBIOS 3.0=0xbfd00000 ACPI 2.0=0xb6760000<br>
>> >>> >> MEMATTR=0xb8973418 RNG=0xbffdbf98<br>
>> >>> >> [ 0.000000] random: fast init done<br>
>> >>> >> [ 0.000000] efi: seeding entropy pool<br>
>> >>> >> :<br>
>> >>> >><br>
>> >>> >> (I am using the latest EDKII master, the Marvell edk2-open-platform<br>
>> >>> >> 17.10<br>
>> >>> >> banch, with all the latest mv-ddr/ atf /etc....).<br>
>> >>> >><br>
>> >>> >> The DT data appear there in die EFI boot, but the PCIe interface<br>
>> >>> >> fails,<br>
>> >>> >> and<br>
>> >>> >> results (I believe) in the CPU stall warnings:<br>
>> >>> >><br>
>> >>> >> [ 717.453025] INFO: rcu_preempt self-detected stall on CPU<br>
>> >>> >> :<br>
>> >>> >> :<br>
>> >>> >> [ 717.589783] armada8k_pcie_probe+0x140/<wbr>0x240<br>
>> >>> >> :<br>
>> >>> >><br>
>> >>> >> Other times, the pcie gets further:<br>
>> >>> >><br>
>> >>> >> [ 3.312127] PCI: OF: host bridge /cp0/pcie@f2600000 ranges:<br>
>> >>> >> [ 3.317740] PCI: OF: IO 0xf9000000..0xf900ffff -> 0xf9000000<br>
>> >>> >> [ 3.323692] PCI: OF: MEM 0xc0000000..0xdfffffff -> 0xc0000000<br>
>> >>> >> [ 3.328915] random: crng init done<br>
>> >>> >> [ 4.326158] armada8k-pcie f2600000.pcie: phy link never came up<br>
>> >>> >> [ 4.332109] armada8k-pcie f2600000.pcie: Link not up after<br>
>> >>> >> reconfiguration<br>
>> >>> >> [ 4.339056] armada8k-pcie f2600000.pcie: PCI host bridge to bus<br>
>> >>> >> 0000:00<br>
>> >>> ><br>
>> >>> ><br>
>> >>> > To be brutally honest, the armada8k-pcie driver is a piece of junk,<br>
>> >>> > and you're much better off using the generic ECAM driver, which now<br>
>> >>> > includes special handling for the missing root port on Synopsys IP.<br>
>> >>> ><br>
>> >>> > It also allows you to have both MMIO32 and MMIO64 regions, which can<br>
>> >>> > be useful with some PCIe cards with large BARs<br>
>> >>> ><br>
>> >>> > Could you try<br>
>> >>> ><br>
>> >>> > compatible = "marvell,armada8k-pcie-ecam";<br>
>> >>> ><br>
>> >>> > in the DT node, please?<br>
>> >>> ><br>
>> >>> > (Before you do that, please check whether UEFI recognizes your PCI<br>
>> >>> > hardware using the 'pci' command in the shell)<br>
>> >>> ><br>
>> >>> ><br>
>> >>> >> [ 4.345705] pci_bus 0000:00: root bus resource [bus 00-ff]<br>
>> >>> >> [ 4.351217] pci_bus 0000:00: root bus resource [io<br>
>> >>> >> 0x0000-0xffff]<br>
>> >>> >> (bus<br>
>> >>> >> address [0xf9000000-0xf900ffff])<br>
>> >>> >> [ 4.360741] pci_bus 0000:00: root bus resource [mem<br>
>> >>> >> 0xc0000000-0xdfffffff]<br>
>> >>> >> [ 4.367659] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400<br>
>> >>> >> [ 4.373708] pci 0000:00:00.0: reg 0x10: [mem<br>
>> >>> >> 0x00000000-0x000fffff<br>
>> >>> >> 64bit]<br>
>> >>> >> [ 4.380562] pci 0000:00:00.0: supports D1 D2<br>
>> >>> >> [ 4.384853] pci 0000:00:00.0: PME# supported from D0 D1 D3hot<br>
>> >>> >> [ 4.390697] pci 0000:00:00.0: bridge configuration invalid ([bus<br>
>> >>> >> 00-00]),<br>
>> >>> >> reconfiguring<br>
>> >>> >> [ 4.398771] pci_bus 0000:01: busn_res: [bus 01-ff] end is<br>
>> >>> >> updated<br>
>> >>> >> to<br>
>> >>> >> 01<br>
>> >>> >> [ 4.405427] pci 0000:00:00.0: BAR 0: assigned [mem<br>
>> >>> >> 0xc0000000-0xc00fffff<br>
>> >>> >> 64bit]<br>
>> >>> >> [ 4.412776] pci 0000:00:00.0: PCI bridge to [bus 01]<br>
>> >>> >> [ 4.725111] pcieport 0000:00:00.0: Signaling PME with IRQ 56<br>
>> >>> >> [ 4.730842] pcieport 0000:00:00.0: AER enabled with IRQ 56<br>
>> >>> >><br>
>> >>> >> but then CPUs are still stalled on some incoming IRQ<br>
>> >>> >><br>
>> >>> >> [ 87.352768] dump_backtrace+0x0/0x150<br>
>> >>> >> [ 87.356445] show_stack+0x14/0x20<br>
>> >>> >> [ 87.359773] sched_show_task+0x14c/0x170<br>
>> >>> >> [ 87.363711] dump_cpu_task+0x40/0x50<br>
>> >>> >> [ 87.367300] rcu_dump_cpu_stacks+0x94/0xd8<br>
>> >>> >> [ 87.371412] rcu_check_callbacks+0x7ac/<wbr>0x980<br>
>> >>> >> [ 87.375700] update_process_times+0x2c/0x58<br>
>> >>> >> [ 87.379900] tick_sched_handle.isra.5+0x30/<wbr>0x50<br>
>> >>> >> [ 87.384449] tick_sched_timer+0x40/0x90<br>
>> >>> >> [ 87.388301] __hrtimer_run_queues+0x124/<wbr>0x198<br>
>> >>> >> [ 87.392676] hrtimer_interrupt+0xe4/0x240<br>
>> >>> >> [ 87.396701] arch_timer_handler_phys+0x30/<wbr>0x40<br>
>> >>> >> [ 87.401163] handle_percpu_devid_irq+0x78/<wbr>0x130<br>
>> >>> >> [ 87.405712] generic_handle_irq+0x24/0x38<br>
>> >>> >> [ 87.409738] __handle_domain_irq+0x5c/0xb8<br>
>> >>> >> [ 87.413850] gic_handle_irq+0x58/0xb0<br>
>> >>> >> [ 87.417526] el1_irq+0xb0/0x128<br>
>> >>> >> [ 87.420678] __do_softirq+0xb0/0x228<br>
>> >>> >> [ 87.424267] irq_exit+0xbc/0xf0<br>
>> >>> >> [ 87.427421] __handle_domain_irq+0x60/0xb8<br>
>> >>> >> [ 87.431533] gic_handle_irq+0x58/0xb0<br>
>> >>> >> [ 87.435209] el1_irq+0xb0/0x128<br>
>> >>> >><br>
>> >>> >> Is anyone aware of any issue like this?<br>
>> >>> >><br>
>> >>> >> Regards,<br>
>> >>> >> Fred<br>
>> >>> >><br>
>> >>> >><br>
>> >>> >> ______________________________<wbr>_________________<br>
>> >>> >> Macchiato mailing list<br>
>> >>> >> <a href="mailto:Macchiato@lists.einval.com">Macchiato@lists.einval.com</a><br>
>> >>> >> <a href="https://lists.einval.com/cgi-bin/mailman/listinfo/macchiato" rel="noreferrer" target="_blank">https://lists.einval.com/cgi-<wbr>bin/mailman/listinfo/macchiato</a><br>
>> >>> ><br>
>> >>> > ______________________________<wbr>_________________<br>
>> >>> > Macchiato mailing list<br>
>> >>> > <a href="mailto:Macchiato@lists.einval.com">Macchiato@lists.einval.com</a><br>
>> >>> > <a href="https://lists.einval.com/cgi-bin/mailman/listinfo/macchiato" rel="noreferrer" target="_blank">https://lists.einval.com/cgi-<wbr>bin/mailman/listinfo/macchiato</a><br>
>> >><br>
>> >><br>
>> ><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div></div></div>