[Macchiato] EDKII grub boot fails with PCIe init

Tue Mar 13 12:36:21 GMT 2018

On 13 March 2018 at 12:26, Frederik Lotter
<frederik.lotter at netronome.com> wrote:
> Hi Ard,
>
> On Mon, Mar 12, 2018 at 6:22 PM, Ard Biesheuvel <ard.biesheuvel at linaro.org>
> wrote:
>>
>> On 12 March 2018 at 16:15, Frederik Lotter
>> <frederik.lotter at netronome.com> wrote:
>> > Hi,
>> > I am getting CPU stall warnings when booting up using the EFI route. I
>> > suspect the PCIe interface, as the stall warning sometimes contain the
>> > probe
>> > function. Other times is seems to get further than PCIe init, but still
>> > stall interrupt handling.
>> > Here are some facts around my observation:
>> >
>> > I have two sdcards for my Machiattobin board. They have identical
>> > kernels
>> > (4.16 rc5) with Ubuntu 16.04 rootfs. The one sdcard uses a uboot, DT and
>> > kernel boot. The second sdcard has EDKII, grub kernel boot. The EDKII
>> > build
>> > includes the device tree DTB (and DTS which I believe is unused) from
>> > the
>> > one used on the uboot sdcard.
>> >
>> > EFI stub: Booting Linux Kernel...
>> > EFI stub: Using DTB from configuration table
>> > EFI stub: Exiting boot services and installing virtual address map...
>> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd081]
>> > [    0.000000] Linux version 4.16.0-rc5-mbcin-netronome-2-dirty
>> > (root at mcb1-cpt) (gcc version 5.4.0 20160609 (Ubuntu/Linaro
>> > 5.4.0-6ubuntu1~16.04.9)) #2 SMP PREEMPT Mon Mar 12 14:40:25 UTC 2018
>> > [    0.000000] Machine model: Marvell 8040 MACHIATOBin
>> > [    0.000000] efi: Getting EFI parameters from FDT:
>> > [    0.000000] efi: EFI v2.70 by EDK II
>> > [    0.000000] efi:  SMBIOS 3.0=0xbfd00000  ACPI 2.0=0xb6760000
>> > MEMATTR=0xb8973418  RNG=0xbffdbf98
>> > [    0.000000] random: fast init done
>> > [    0.000000] efi: seeding entropy pool
>> > :
>> >
>> > (I am using the latest EDKII master, the Marvell edk2-open-platform
>> > 17.10
>> > banch, with all the latest mv-ddr/ atf /etc....).
>> >
>> > The DT data appear there in die EFI boot, but the PCIe interface fails,
>> > and
>> > results (I believe) in the CPU stall warnings:
>> >
>> > [  717.453025] INFO: rcu_preempt self-detected stall on CPU
>> > :
>> > :
>> > [  717.589783]  armada8k_pcie_probe+0x140/0x240
>> > :
>> >
>> > Other times, the pcie gets further:
>> >
>> > [    3.312127] PCI: OF: host bridge /cp0/pcie at f2600000 ranges:
>> > [    3.317740] PCI: OF:    IO 0xf9000000..0xf900ffff -> 0xf9000000
>> > [    3.323692] PCI: OF:   MEM 0xc0000000..0xdfffffff -> 0xc0000000
>> > [    3.328915] random: crng init done
>> > [    4.326158] armada8k-pcie f2600000.pcie: phy link never came up
>> > [    4.332109] armada8k-pcie f2600000.pcie: Link not up after
>> > reconfiguration
>> > [    4.339056] armada8k-pcie f2600000.pcie: PCI host bridge to bus
>> > 0000:00
>>
>>
>> To be brutally honest, the armada8k-pcie driver is a piece of junk,
>> and you're much better off using the generic ECAM driver, which now
>> includes special handling for the missing root port on Synopsys IP.
>>
>> It also allows you to have both MMIO32 and MMIO64 regions, which can
>> be useful with some PCIe cards with large BARs
>>
>> Could you try
>>
>> compatible = "marvell,armada8k-pcie-ecam";
>>
>> in the DT node, please?
>>
>> (Before you do that, please check whether UEFI recognizes your PCI
>> hardware using the 'pci' command in the shell)
>
>
> This exercise help a lot. Thank you for the proposal.
>
> So now I can consistently boot using uboot and efi.
>
> However, the pcie driver init fails. I have provided boot logs and also my
> DT entry - we need custom BAR ranges, and I am not sure if this driver
> understand everything.
>
>  cp0_pcie0: pcie at f2600000 {
>   compatible = "marvell,armada8k-pcie-ecam", "snps,dw-pcie";
>   reg = <0 0xf2600000 0 0x10000>,
>         <0 ((0xf6000000 + (0 * 0x1000000)) + 0xf00000) 0 0x80000>;
>   reg-names = "ctrl", "config";
>   #address-cells = <3>;
>   #size-cells = <2>;
>   #interrupt-cells = <1>;
>   device_type = "pci";
>   dma-coherent;
>   msi-parent = <&gic_v2m0>;
>
>   bus-range = <0 0xff>;
>   ranges =
>
>   <0x81000000 0 (0xf9000000 + (0 * 0x10000)) 0 (0xf9000000 + (0 * 0x10000))
> 0 0x10000
>
>   0x82000000 0 (0xf6000000 + (0 * 0x1000000)) 0 (0xf6000000 + (0 *
> 0x1000000)) 0 0xf00000>;
>   interrupt-map-mask = <0 0 0 0>;
>   interrupt-map = <0 0 0 0 &cp0_icu 0x0 22 4>;
>   interrupts = <0x0 22 4>;
>   num-lanes = <1>;
>   clocks = <&cp0_clk 1 13>;
>   status = "disabled";
>  };
>
>
> Error:
>
> [    1.396968] PCI: OF: host bridge /cp0/pcie at f2600000 ranges:
> [    1.396979] PCI: OF:    IO 0xf9000000..0xf900ffff -> 0xf9000000
> [    1.396984] PCI: OF:   MEM 0xc0000000..0xdfffffff -> 0xc0000000
> [    1.396998] pci-host-generic f2600000.pcie: ECAM area [mem
> 0xf2600000-0xf260ffff] can only accommodate [bus 00-ffffffffffffffff]
> (reduced from [bus 00-ff] desired)
> [    1.397002] pci-host-generic f2600000.pcie: ECAM ioremap failed
> [    1.397011] pci-host-generic: probe of f2600000.pcie failed with error
> -12
>
>
> Thanks for the support.
>

Please try the following config

cp0_pcie0: pcie at e0000000 {
   compatible = "marvell,armada8k-pcie-ecam", "snps,dw-pcie";
   reg = <0 0xe0000000 0 0xff00000>;
   #address-cells = <3>;
   #size-cells = <2>;
   #interrupt-cells = <1>;
   device_type = "pci";
   dma-coherent;
   msi-parent = <&gic_v2m0>;

   bus-range = <0 0xfe>;
   ranges = <0x1000000 0x0 0x00000000 0x0 0xeff00000 0x0 0x00010000>,
            <0x2000000 0x0 0xc0000000 0x0 0xc0000000 0x0 0x20000000>,
            <0x3000000 0x8 0x00000000 0x8 0x00000000 0x1 0x00000000>;

   interrupt-map-mask = <0 0 0 0>;
   interrupt-map = <0 0 0 0 &cp0_icu 0x0 22 4>;
};