[Macchiato] MacchiatoBin GPU progress [#freetimeproject]

Mon Jun 5 19:55:31 BST 2017

Hi,

You could try putting something in the path that is actively modifying signals as well. A PCIe switch/expansion chassis comes to mind.

Cheap/cheezy choice:
https://www.amazon.com/Switch-Multiplier-Expander-Riser-Expansion/dp/B01ERXUEOM
(although they are about 1/2 that price on ebay)

If you can get the switch working, then you might be able to plug your card into that.

More spendy alternatives, for people that think routing PCIe over a USB cable is a bad idea (although It actually seems to work).

https://www.amazon.com/StarTech-Express-Expansion-Enclosure-System/dp/B002I9SK5S/ref=pd_sbs_147_1?_encoding=UTF8&pd_rd_i=B002I9SK5S&pd_rd_r=HP99Q6HVWDYC2BS8VKMV&pd_rd_w=DPiPe&pd_rd_wg=c2xie&psc=1&refRID=HP99Q6HVWDYC2BS8VKMV

(even more spendy, although generally robust enough for real use)

http://www.onestopsystems.com/pcie-expansion-components
http://cyclone.com/products/expansion_systems/index.php
http://magma.com/products/pcie-expansion/expressbox-4-1u/

Or the dozens of other choices...

|-----Original Message-----
|From: Macchiato [mailto:macchiato-bounces at lists.einval.com] On Behalf Of
|Ard Biesheuvel
|Sent: Monday, June 05, 2017 7:10 AM
|To: Matt Spencer
|Cc: Steve McIntyre; macchiato at lists.einval.com; Steve Capper; Leif
|Lindholm; Wookey
|Subject: Re: [Macchiato] MacchiatoBin GPU progress [#freetimeproject]
|
|On 5 June 2017 at 12:01, Matt Spencer <Matt.Spencer at arm.com> wrote:
|> Hey Ard
|>
|>
|> Just wanted to touch base on this issue - has there been any progress
|> beyond the patches you suggested in your last mail?  Are we any closer
|> to an official solution?  And is there something that I am able to try
|> out on my hardware yet?
|>
|
|Hello Matt,
|
|I managed to depopulate the JTAG header on my board, and gently hack
|away the back of the PCIe connector, so I can plug x8 and x86 cards, and
|eliminate the riser from the equation.
|
|As it turns out, this does not make a whole lot of difference: I have a x1
|Realtek 8169 NIC that is rock solid, a couple of x1 XHCI cards that are so so,
|and beyond that, no x8 or x16 cards can train successfully at all. This
|strengthens my suspicion that there are either signal integrity or power
|routing issues with this board.
|
|Does your GPU card's link train successfully every time? And does your riser
|have a separate power lead connected?
|
|I have asked Semihalf and Marvell for feedback regarding the types and
|brands of PCIe cards that they tested, but given the insufficient size of the
|PCIe MMIO windows, it is pretty obvious that they never tried any graphics in
|the first place.
|
|We (as Linaro, or ARM for that matter) are not really in a position to debug
|this. These issues may be corrected with increased drive strength settings, or
|other board integration level controls on the SoC that we have no
|documentation for, nor the means to discover the correct values. So the ball
|is really in Marvell's court on this one, I'm afraid.
|
|--
|Ard.
|
|> ________________________________
|> From: Ard Biesheuvel <ard.biesheuvel at linaro.org>
|> Sent: 06 May 2017 09:14:29
|> To: Matt Spencer
|> Cc: Leif Lindholm; Wookey; Steve Capper; Steve McIntyre;
|> macchiato at lists.einval.com
|> Subject: Re: MacchiatoBin GPU progress [#freetimeproject]
|>
|> On 5 May 2017 at 18:02, Matt Spencer <Matt.Spencer at arm.com> wrote:
|>> Hi Ard, Leif has suggested I add you to this thread.
|>>
|>>
|>> I am trying to get the MacchiatoBin working with a PCIe GPU.  The
|>> approach I am taking is to use a powered 16->1 lane PCIe adapter
|>>
|>>
|(https://www.amazon.co.uk/gp/product/B01ER2Z1GY/ref=oh_aui_detailpage
|>> _o03_s00?ie=UTF8&psc=1) and using a modern NVidia GForce GPU that is
|>> know to function well with Nouveau
|>>
|>>
|(https://www.amazon.co.uk/gp/product/B01AY7927A/ref=oh_aui_detailpag
|e_o04_s00?ie=UTF8&psc=1).
|>>
|>>
|>> I have the kernel booting, detecting the graphics card and trying to
|>> initialise the driver, but come across this issue:
|>>
|>
|> ...
|>> root at localhost:~# lspci
|>>
|>> 00:00.0 PCI bridge: Marvell Technology Group Ltd. Device 0110
|>>
|>> 01:00.0 VGA compatible controller: NVIDIA Corporation Device 128b
|>> (rev a1)
|>>
|>> 01:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio
|>> Controller (rev
|>> a1)
|>>
|>>
|>>
|>> It looks like I don't have enough BAR memory available?  Leif tells
|>> me that the base configuration only exposes 15M, but you have a
|>> different firmware that could solve the problem?
|>>
|>
|> Correct. The firmware configuration of this board is slightly silly,
|> but can be improved a lot by reconfiguring the translation windows at
|> various levels to allow for a much larger BAR space (512 MB + 4GB
|> rather than 15 MB, which is simply insufficient for graphics cards)
|>
|> The problem is that, while these changes could easily be applied to
|> the Marvell version of u-boot, we currently only have a firmware stack
|> based on UEFI and the upstream version of the device tree (rather than
|> the wildly incompatible Marvell one), and there are other issues we
|> need to sort out before this stack is usable enough (i.e., you will
|> lose your network connection if you switch to it now)
|>
|> As for the stability issues we identified: our goal is to produce a
|> firmware stack for this board that allows you to boot bog standard
|> distro installers, whose kernels trigger this issue, and currently,
|> the only workaround that is guaranteed to prevent this issue (even for
|> KVM guests) is to take down two of the four cores in the firmware.
|> Graphics card support is high on our list as well, but it will simply
|> take us some time to get all of this sorted.
|>
|> In the mean time, you are welcome to have a look at the required
|> changes on the ATF side: this patch
|>
|> https://github.com/ardbiesheuvel/arm-trusted-
|firmware/commit/591d1cb7c
|> 3bd4fa807093643ef8300d15973818e
|>
|> is the only one you will need for ATF. Next, you will need to add the
|> window 0xc0000000-0xdfffffff to the PCIe DT node, which should be
|> sufficient to get the driver to allocate BARs from it. However, given
|> that this range was formerly in use for DRAM, the DT description of
|> memory needs to be updated as well, and this slice removed. (In the
|> UEFI code, I actually remap it so you don't lose it, but just hiding
|> it from the kernel should be sufficient in the short term)
|>
|> Hope this helps,
|> Ard.
|> IMPORTANT NOTICE: The contents of this email and any attachments are
|> confidential and may also be privileged. If you are not the intended
|> recipient, please notify the sender immediately and do not disclose
|> the contents to any other person, use it for any purpose, or store or
|> copy the information in any medium. Thank you.
|
|_______________________________________________
|Macchiato mailing list
|Macchiato at lists.einval.com
|https://lists.einval.com/cgi-bin/mailman/listinfo/macchiato
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.