<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi Ard, Leif has suggested I add you to this thread.</p>
<p><br>
</p>
<p>I am trying to get the MacchiatoBin working with a PCIe GPU. The approach I am taking is to use a powered 16->1 lane PCIe adapter (<a href="https://www.amazon.co.uk/gp/product/B01ER2Z1GY/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1" class="x_OWAAutoLink">https://www.amazon.co.uk/gp/product/B01ER2Z1GY/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1</a>)
and using a modern NVidia GForce GPU that is know to function well with Nouveau (<a href="https://www.amazon.co.uk/gp/product/B01AY7927A/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1" class="x_OWAAutoLink">https://www.amazon.co.uk/gp/product/B01AY7927A/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1</a>).</p>
<p><br>
</p>
<p>I have the kernel booting, detecting the graphics card and trying to initialise the driver, but come across this issue:</p>
<p><br>
</p>
<p></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048239] PCI host bridge /cpn-110-master/pcie@0x600000 ranges:</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048252] IO 0xf9000000..0xf900ffff -> 0xf9000000</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048257] MEM 0xf6000000..0xf6efffff -> 0xf6000000</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048403] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048407] pci_bus 0000:00: root bus resource [bus 00-ff]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048412] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] (bus address [0xf9000000-)</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048415] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.048623] iommu: Adding device 0000:00:00.0 to group 0</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.074114] pci 0000:01:00.0: reg 0x24: initial BAR value 0x00000000 invalid</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.074772] iommu: Adding device 0000:01:00.0 to group 1</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.074786] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=none,locks=none</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.075869] iommu: Adding device 0000:01:00.1 to group 1</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084112] pci 0000:00:00.0: BAR 15: no space for [mem size 0x0c000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084116] pci 0000:00:00.0: BAR 15: failed to assign [mem size 0x0c000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084119] pci 0000:00:00.0: BAR 14: no space for [mem size 0x01800000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084121] pci 0000:00:00.0: BAR 14: failed to assign [mem size 0x01800000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084125] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6000000-0xf60fffff]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084129] pci 0000:00:00.0: BAR 13: assigned [io 0x1000-0x1fff]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084135] pci 0000:01:00.0: BAR 1: no space for [mem size 0x08000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084137] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x08000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084140] pci 0000:01:00.0: BAR 3: no space for [mem size 0x02000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084142] pci 0000:01:00.0: BAR 3: failed to assign [mem size 0x02000000 64bit pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084144] pci 0000:01:00.0: BAR 0: no space for [mem size 0x01000000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084147] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x01000000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084150] pci 0000:01:00.0: BAR 6: no space for [mem size 0x00080000 pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084152] pci 0000:01:00.0: BAR 6: failed to assign [mem size 0x00080000 pref]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084154] pci 0000:01:00.1: BAR 0: no space for [mem size 0x00004000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084157] pci 0000:01:00.1: BAR 0: failed to assign [mem size 0x00004000]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084159] pci 0000:01:00.0: BAR 5: assigned [io 0x1000-0x107f]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084182] pci 0000:00:00.0: PCI bridge to [bus 01]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084186] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084428] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084431] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">[ 3.084432] pci 0000:01:00.1: Signaling PME through PCIe PME interrupt</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style=""><br>
</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols; font-size:16px">For reference, PCI devices on the bus are:</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols; font-size:16px"><br>
</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="font-family:Calibri,Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols; font-size:16px"><span style="font-family:Calibri,Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols; font-size:16px"></span></span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">root@localhost:~# lspci</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">00:00.0 PCI bridge: Marvell Technology Group Ltd. Device 0110</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">01:00.0 VGA compatible controller: NVIDIA Corporation Device 128b (rev a1)</span></p>
<p style="margin:0px; font-style:normal; font-weight:normal; font-size:11px; line-height:normal; font-family:Menlo">
<span style="">01:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)</span></p>
<p></p>
<p>It looks like I don't have enough BAR memory available? Leif tells me that the base configuration only exposes 15M, but you have a different firmware that could solve the problem?</p>
<p><br>
</p>
<p>I am going to cross post this to the list that Steve M setup to track Macchiato activities.</p>
<p><br>
</p>
<p>/Matt</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Leif Lindholm<br>
<b>Sent:</b> 05 May 2017 16:20:15<br>
<b>To:</b> Matt Spencer; Wookey; Steve Capper; Steve McIntyre<br>
<b>Subject:</b> Re: MacchiatoBin GPU progress [#freetimeproject]</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">This is a firmware change.<br>
<br>
We're currently a bit distracted trying to get to the bottom of the <br>
random TLB failures with > 39 bits VA.<br>
<br>
But if you don't mind adding Ard (.Biesheuvel@linaro.org) to the thread, <br>
we can keep you posted. (And we need to be writing some stuff on the <br>
Linaro wiki.)<br>
<br>
/<br>
Leif<br>
<br>
On 05/05/17 15:53, Matt Spencer wrote:<br>
> I bought a low end new card - a Geforce GT710. It is a 16 lane PCIe<br>
> device, so I also picked up a 16->1 powered PCIe adapter. The plan was<br>
> to upgrade this to a 16->4 adapter to increase the available bandwidth<br>
> if this experiment worked.<br>
><br>
><br>
> <a href="https://www.amazon.co.uk/gp/product/B01ER2Z1GY/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1">
https://www.amazon.co.uk/gp/product/B01ER2Z1GY/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1</a><br>
><br>
> <a href="https://www.amazon.co.uk/gp/product/B01AY7927A/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1">
https://www.amazon.co.uk/gp/product/B01AY7927A/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1</a><br>
><br>
><br>
> Is the change a runtime/configuration change or is it a kernel<br>
> build-time configuration? And where can I get more details?<br>
><br>
><br>
> /Matt<br>
><br>
> ------------------------------------------------------------------------<br>
> *From:* Leif Lindholm<br>
> *Sent:* 05 May 2017 15:35:22<br>
> *To:* Matt Spencer; Wookey; Steve Capper; Steve McIntyre<br>
> *Subject:* Re: MacchiatoBin GPU progress [#freetimeproject]<br>
><br>
> Well, the default configuration only gives you 15MB of mappable BAR space.<br>
><br>
> Ard has a version that gives you gigabytes, with a completely different<br>
> setup than the Marvell kernel. We can share this.<br>
><br>
> However, you seem to be doing better in one regard - most of our cards<br>
> don't show up at all. Which specific GForce card is this you're using?<br>
><br>
> /<br>
> Leif<br>
><br>
> On 05/05/17 15:22, Matt Spencer wrote:<br>
>> Hi Guys<br>
>><br>
>><br>
>> Getting closer (one tiny step at a time!). The GForce GPU is detected<br>
>> on the PCIe bus, and the Nouveau driver loaded - but then this happens:<br>
>><br>
>><br>
>> [ 3.093228] PCI host bridge /cpn-110-master/pcie@0x600000 ranges:<br>
>><br>
>> [ 3.099371] IO 0xf9000000..0xf900ffff -> 0xf9000000<br>
>><br>
>> [ 3.104539] MEM 0xf6000000..0xf6efffff -> 0xf6000000<br>
>><br>
>> [ 3.109819] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00<br>
>><br>
>> [ 3.116471] pci_bus 0000:00: root bus resource [bus 00-ff]<br>
>><br>
>> [ 3.121989] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]<br>
>> (bus address [0xf9000000-)<br>
>><br>
>> [ 3.131514] pci_bus 0000:00: root bus resource [mem<br>
>> 0xf6000000-0xf6efffff]<br>
>><br>
>> [ 3.138640] iommu: Adding device 0000:00:00.0 to group 0<br>
>><br>
>> [ 24.152049] INFO: rcu_preempt detected stalls on CPUs/tasks:<br>
>><br>
>> [ 24.157748] 1-...: (1 GPs behind) idle=941/140000000000000/0<br>
>> softirq=145/146 fqs=5251<br>
>><br>
>> [ 24.165784] (detected by 2, t=5255 jiffies, g=-223, c=-224, q=13)<br>
>><br>
>> [ 24.172002] Task dump for CPU 1:<br>
>><br>
>> [ 24.175242] kworker/u8:0 R running task 0 6 2<br>
>> 0x00000002<br>
>><br>
>> [ 24.182356] Workqueue: deferwq deferred_probe_work_func<br>
>><br>
>> [ 24.187612] Call trace:<br>
>><br>
>> [ 24.190072] [<ffffffc000086cc4>] __switch_to+0x8c/0xa0<br>
>><br>
>> [ 24.195234] [<ffffffc0000cc284>] __queue_work+0x124/0x2a0<br>
>><br>
>> [ 24.200656] [<ffffffc0000ecf74>] __wake_up+0x44/0x68<br>
>><br>
>> [ 24.205642] [<ffffffc0000c9be0>] call_usermodehelper_exec+0xb8/0x178<br>
>><br>
>> [ 24.212021] [<ffffffc0de893330>] 0xffffffc0de893330<br>
>><br>
>><br>
>> To my mind, it looks like the kernel stalls when mapping the IO for the<br>
>> graphics card - but its been a number of years since I hacked on the<br>
>> kernel - do any of you have any ideas where to look next?<br>
>><br>
>><br>
>> /Matt<br>
>><br>
><br>
<br>
</div>
</span></font>IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use
it for any purpose, or store or copy the information in any medium. Thank you.
</body>
</html>