Difference between revisions of "APF6 SP The full howto"

From ArmadeusWiki
Jump to: navigation, search
(→‎Adding the button and led (PIO))
(→‎Generate bitstream with Quartus)
 
(106 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
[[Category: APF6]]
 
[[Category: APF6]]
 +
[[Category: APF6_SP]]
 
[[Category: FPGA]]
 
[[Category: FPGA]]
 
[[Category: CycloneV]]
 
[[Category: CycloneV]]
 +
[[Category: Quartus]]
 +
[[Category: Qsys]]
  
−
{{Under_Construction}}
+
== Introduction ==
  
−
= Introduction =
+
This is the full howto make an APF6_SP CycloneV example design which uses :
−
 
+
−
This is the full howto to make an APF6_SP CycloneV example design to use :
+
 
* PCIe
 
* PCIe
 
* DDR3
 
* DDR3
−
* LED
+
* General Purpose Input/Outputs
−
* Button
+
** LED
 +
** Button
  
 
This howto is designed for :
 
This howto is designed for :
−
* APF6D_SP-M1GE4G-4m-BW-C4M384: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips
+
* [[APF6_SP|APF6D_SP-M1GE4G-4m-BW-C4M384]]: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips
−
* Quartus 15.1.1 : Quartus 15 with update 1
+
* Quartus with following versions:
 +
** '''Quartus 17.0.0''' : Quartus 17 without update
 +
** '''Quartus 19.1''' : Quartus Intel version
 +
** To install Quartus see [[Quartus_installation_on_Linux | this page]].
  
 
[[File:APF6_SP_full_howto.png|frame|center|The full design for this howto]]
 
[[File:APF6_SP_full_howto.png|frame|center|The full design for this howto]]
−
 
−
To install quartus 15 see [[Quartus_installation_on_Linux#Version_15.1_prime_lite | this page]].
 
  
 
= Make the quartus project =
 
= Make the quartus project =
  
−
* Open the new project wizard in Quartus
+
* Open the "New Project Wizard" in Quartus
−
* Select a directory for the design:
+
* Select a directory for the design, for example ''/home/your_name/workspace'' and a project name ''apf6_the_full_howto'':
−
<pre class="config">
+
* Then Choose ''Empty Project'' and don't add any file yet -> press "Next"
−
~/tmp/apf6_the_full_howto
+
* Select the part number : '''5CGXFC4C7U19C8''' -> press "Next" -> "Next" -> "Finish"
−
</pre>
+
* Right click on "Cyclone V -> Devices..." in project navigator and click on '''Device and Pin Options'''  
−
* Open an empty project
+
* in '''CvP Settings''' set this options:
−
* Change the device : right click on "Cyclone V"->'''Devices...''' in Project Navigator
+
** Configuration via Protocol: '''Core initialization and update'''
−
* Select the part number : '''5CGXFC4C7U19C8''' then '''Ok'''
+
* then select "OK" two times
  
 
= Make the Qsys project =
 
= Make the Qsys project =
  
−
* Open Qsys: '''Tools'''->'''Qsys'''
+
* Open Qsys/Platform designer:
−
* Delete the component '''clk_0''' then save the Qsys project with name : '''qsys_tfht.qsys'''
+
** With Quartus 17: '''Tools'''->'''Qsys'''
−
* Clic on '''finish''' and generate the component with all options left by default
+
** With Quartus 19.1: '''Tools'''->'''Platform designer'''
−
* Under Quartus project add the Qsys project (don't forget to click on '''add''' after selected here.
+
* Delete the component '''clk_0''' then save the Qsys project as : '''qsys_tfht.qsys'''
 +
* Click on '''Finish''' and generate the component with all options left by default
 +
* Under Quartus project add the Qsys project (don't forget to click on '''add''' after selected here).
 
[[File:tfht_add_qsys_project.png|frame|center|How to add a file in project]]
 
[[File:tfht_add_qsys_project.png|frame|center|How to add a file in project]]
−
* Relaunch Qsys by double-click on the file in project navigator
+
* Right click on ''qsys_tfht.qsys'' and '''set as Top-Level Entity'''
 +
* Relaunch Qsys by double-clicking on the file in "Project Navigator"
  
 
== Adding the PCIe and CvP ==
 
== Adding the PCIe and CvP ==
Line 48: Line 54:
 
     Interface Protocols
 
     Interface Protocols
 
         PCI Express
 
         PCI Express
−
             Avalon-MM Cyclone V Hard IP for PCI Express
+
             Avalon-MM Cyclone V Hard IP for PCI Express  
 +
            Avalon-MM Cyclone V Hard IP for PCI Express Intel FPGA IP  <- 19.1
 
</pre>
 
</pre>
 +
* "Add" it to design.
 
* PCIe configuration:
 
* PCIe configuration:
 
** Number of Lanes: '''x1'''
 
** Number of Lanes: '''x1'''
 
** Reference clock frequency: '''125Mhz'''
 
** Reference clock frequency: '''125Mhz'''
−
** check "Use 62.5 MHz application clock"
+
** check '''Use 62.5 MHz application clock'''
 +
** check ''' Enable configuration via the PCIe link'''
 
** BAR0: 32-bit non-prefetchable memory
 
** BAR0: 32-bit non-prefetchable memory
 
** BAR1: 32-bit non-prefetchable memory
 
** BAR1: 32-bit non-prefetchable memory
Line 61: Line 70:
 
** Class Code: '''0x00001300''': there is a bug in quartus, the actual class code is here '''13'''
 
** Class Code: '''0x00001300''': there is a bug in quartus, the actual class code is here '''13'''
  
−
* click on finish
+
* click on "Finish"
−
* Export these signal with name in '''bold''':
+
* Export these signals with name in '''bold''' (right click in "Hierarchy", then choose "Connections" and "Export as:..."). Exported signals will be accessible on top of module:
 
** refclk : '''refclk'''
 
** refclk : '''refclk'''
 
** npor: '''npor'''
 
** npor: '''npor'''
 
** hip_serial: '''hip_serial'''
 
** hip_serial: '''hip_serial'''
−
* connect '''Rxm_BAR0''' on '''Cra'''
+
* connect '''Rxm_BAR0''' on '''Cra''' (always right click in "Hierarchy", then "Connections"...)
  
−
* Adding CvP component reconfig driver, with all default parameters:
+
* Add CvP component reconfig driver, with all default parameters:
 
<pre class="config">
 
<pre class="config">
 
Library
 
Library
Line 74: Line 83:
 
         PCI Express
 
         PCI Express
 
             Altera PCIe Reconfig Driver
 
             Altera PCIe Reconfig Driver
 +
            Altera PCIe Reconfig Driver Intel FPGA IP <- 19.1
 
</pre>
 
</pre>
−
* Connect these signals:
+
* Connect previous IP signals with:
 
** '''coreclkout''' -> '''reconfig_xcvr_clk'''
 
** '''coreclkout''' -> '''reconfig_xcvr_clk'''
 
** '''coreclkout''' -> '''pld_clk'''
 
** '''coreclkout''' -> '''pld_clk'''
 
** '''nreset_status''' -> '''reconfig_xcvr_rst'''
 
** '''nreset_status''' -> '''reconfig_xcvr_rst'''
  
−
* Adding CvP component transceiver, with all default parameters:
+
* Add CvP component transceiver, with all default parameters:
 
<pre class="config">
 
<pre class="config">
 
Library
 
Library
Line 86: Line 96:
 
         Transceiver PHY
 
         Transceiver PHY
 
             Transceiver Reconfiguration Controller
 
             Transceiver Reconfiguration Controller
 +
            Transceiver Reconfiguration Controller Intel FPGA IP <- 19.1
 
</pre>
 
</pre>
−
* Connect these signals:
+
* Connect previous PCIe IP signals with:
 
** '''coreclkout''' -> '''mgmt_clk_clk'''
 
** '''coreclkout''' -> '''mgmt_clk_clk'''
 
** '''nreset_status''' -> '''mgmt_rst_reset'''
 
** '''nreset_status''' -> '''mgmt_rst_reset'''
−
** '''reconfig_mgmt''' -> '''reconfig_mgmt'''
+
** reconfig_driver.'''reconfig_mgmt''' -> '''reconfig_mgmt'''
 
** reconfig_driver.'''reconfig_busy''' -> '''reconfig_busy'''
 
** reconfig_driver.'''reconfig_busy''' -> '''reconfig_busy'''
 
** pci.'''reconfig_to_xcvr''' -> '''reconfig_to_xcvr'''
 
** pci.'''reconfig_to_xcvr''' -> '''reconfig_to_xcvr'''
 
** pci.'''reconfig_from_xcvr''' -> '''reconfig_from_xcvr'''
 
** pci.'''reconfig_from_xcvr''' -> '''reconfig_from_xcvr'''
  
 +
* you should end up with the following wiring:
 
[[File:APF6_tfht_pcie_wiring.png|frame|center|Wiring result after PCIe configured]]
 
[[File:APF6_tfht_pcie_wiring.png|frame|center|Wiring result after PCIe configured]]
−
 
  
 
== Adding the DDR3 ==
 
== Adding the DDR3 ==
  
−
{{Note| When parameters should be left by default they are not indicated}}
+
{{Note| If not indicated, parameters should be left to default one !}}
  
 
=== UniPHY controller ===
 
=== UniPHY controller ===
Line 110: Line 121:
 
         Memory Interfaces with UniPHY
 
         Memory Interfaces with UniPHY
 
             DDR3 SDRAM Controller with UniPHY
 
             DDR3 SDRAM Controller with UniPHY
 +
            DDR3 SDRAM Controller with UniPHY Intel FPGA IP <- 19.1
 
</pre>
 
</pre>
  
Line 121: Line 133:
 
** supply Voltage: '''1.35V DDR3L'''
 
** supply Voltage: '''1.35V DDR3L'''
 
* In tab '''Memory Parameters'''
 
* In tab '''Memory Parameters'''
−
** In preset select: '''JEDEC DDR3-1G6 2GB X8''' and apply
+
** In "Presets" (right of the window) select: '''JEDEC DDR3-1G6 2GB X8''' and apply
 
** Memory device speed grade: '''400Mhz'''
 
** Memory device speed grade: '''400Mhz'''
 
** Total interface width: '''24'''
 
** Total interface width: '''24'''
Line 128: Line 140:
 
* In tab '''Controller Settings'''
 
* In tab '''Controller Settings'''
 
** checkbox: '''Generate power-of-2 data bus widths for Qsys or SOPC Builder'''
 
** checkbox: '''Generate power-of-2 data bus widths for Qsys or SOPC Builder'''
−
** checkbox: '''Enable Configuration and Status Register Interface (Internacl JTAG)'''
+
** checkbox: '''Enable Configuration and Status Register Interface''' (Internal JTAG)
 
** checkbox: '''Enable Error Detection and Correction Logic'''
 
** checkbox: '''Enable Error Detection and Correction Logic'''
  
−
* Click on ''finish''
+
* Click on ''Finish''
  
 
* Connect following signals:
 
* Connect following signals:
Line 148: Line 160:
 
=== Span extender ===
 
=== Span extender ===
  
−
DDR3 memory is to large for PCIe BAR domain. Then we have to use a span extender to adapt PCIe BAR domain to DDR3 domain.
+
DDR3 memory is to large for PCIe BAR domain. Then we have to use a [https://documentation.altera.com/#/00030683-AA$NT00064780 span extender] to adapt PCIe BAR domain to DDR3 domain.
  
 
* Select span expander in IP Library:
 
* Select span expander in IP Library:
Line 161: Line 173:
 
** Datapath Width: '''64bits'''
 
** Datapath Width: '''64bits'''
 
** Slave Word Address Width: '''20bits'''
 
** Slave Word Address Width: '''20bits'''
−
** Finish
+
** press "Finish"
  
−
* Connect following extender signals:
+
* Connect following PCIe IP signals to extender one:
 
** '''coreclckout''' -> '''clock'''
 
** '''coreclckout''' -> '''clock'''
 
** '''nreset_status''' -> '''reset'''
 
** '''nreset_status''' -> '''reset'''
−
** '''Rxm_BAR1''' -> '''windowed_slave'''
 
−
** '''expanded_master''' -> '''avl_0'''
 
 
** '''Rxm_BAR0''' -> '''cntl'''
 
** '''Rxm_BAR0''' -> '''cntl'''
 
*** Change base address of cntl to '''0x0000_4000'''
 
*** Change base address of cntl to '''0x0000_4000'''
 +
** '''Rxm_BAR1''' -> '''windowed_slave'''
 +
* And following extender signals to DDR3 one:
 +
** '''expanded_master''' -> mem_if_ddr3_emif_0.'''avl_0'''
 +
 +
== Adding the button and LED (PIO) ==
  
−
== Adding the button and led (PIO) ==
+
Documentation of the PIO component can be found [https://www.altera.co.jp/content/dam/altera-www/global/ja_JP/pdfs/literature/hb/nios2/n2cpu_nii51007.pdf here].
  
 
* In IP Library select:
 
* In IP Library select:
Line 179: Line 194:
 
         Peripherals
 
         Peripherals
 
             PIO (Parallel I/O)
 
             PIO (Parallel I/O)
 +
            PIO (Parallel I/O) Intel FPGA IP <- 19.1
 
</pre>
 
</pre>
 
* Configure it with these options:
 
* Configure it with these options:
 +
** Width: '''2'''
 
** Direction: '''Bidir'''
 
** Direction: '''Bidir'''
 
** checkbox '''Synchronously capture'''
 
** checkbox '''Synchronously capture'''
Line 186: Line 203:
 
** checkbox '''Generate IRQ'''
 
** checkbox '''Generate IRQ'''
 
** IRQ Type: '''EDGE'''
 
** IRQ Type: '''EDGE'''
−
* click on finish
+
* click on "Finish"
  
−
* export ''external_connection'' to '''pio'''
+
* export ''pio.external_connection'' to '''pio'''
  
 
* Connect following signals:
 
* Connect following signals:
−
** '''coreclkout''' -> '''clk'''
+
** '''coreclkout''' -> '''pio_0.clk'''
−
** '''nreset_status''' -> '''reset'''
+
** '''nreset_status''' -> '''pio_0.reset'''
−
** '''Rxm_BAR2''' -> '''s1'''
+
** '''Rxm_BAR2''' -> '''pio_0.s1'''
−
** '''irq''' -> '''Rxmirq'''
+
** '''pio_0.irq''' -> '''Rxmirq'''
  
 
== Qsys component schematic ==
 
== Qsys component schematic ==
Line 202: Line 219:
 
[[File:apf6_tfht_pcieddr_sch.png|frame|center| The full connections in qsys component]]
 
[[File:apf6_tfht_pcieddr_sch.png|frame|center| The full connections in qsys component]]
  
−
= Back to quartus project configuration =
+
Now click on '''finish''' and generate the component with all default parameters kept.
 +
 
 +
= Generate bitstream with Quartus =
 +
 
 +
* Make a first synthesis, in ''Task'' window, right click on ''Analysis & Synthesis'' then choose ''Start''
 +
<pre class="config">
 +
Compile Design
 +
    + Analysis & Synthesis
 +
</pre>
 +
{{Note| If you have a synthesis error here with 19.1 be sure that you are using your [[Quartus_installation_on_Linux#Perl_Getopt::Long | perl distribution]].}}
 +
== Pinout ==
 +
 
 +
* PCIe
 +
** Outside Quartus, create a file named '''pcie_pinout.tcl'' and edit it with your favorite text editor.
 +
** Copy all lines from [[APF6_SP_DDR3_PINOUT#PCIe| here]].
 +
** Save your file, then add it to the project inside Quartus
 +
** Run the script with following menu, select it then '''run''':
 +
<pre class="config">
 +
Tools
 +
    TCL Scripts ...
 +
</pre>
 +
* DDR3
 +
** Create a file named '''ddr3_pinout.tcl''' and edit it.
 +
** Copy all lines from [[APF6_SP_DDR3_PINOUT#DDR3| here]] ('''Pinout placement''' and '''Technology pinout''').
 +
** Save then add the file to project
 +
** Run the script with following menu, select it then '''run''':
 +
<pre class="config">
 +
Tools
 +
    TCL Scripts ...
 +
</pre>
 +
* Button & Led
 +
** Open the pinplanner by double-click in '''tasks''':
 +
<pre class="config">
 +
Compile Design
 +
    Analysis & Synthesis
 +
        I/O Assignment Analysis
 +
            Pin Planner
 +
</pre>
 +
** For button '''pio_export[0]''' select Location column and set it to : '''PIN_L9'''
 +
** For led '''pio_export[1]''' select Location column and set it to : '''PIN_M10'''
 +
** Close the Pin Planner
 +
 
 +
== Synthesis, Place & route ==
 +
 
 +
* Right click on ''Analysis and Synthesis'' then '''Start'''
 +
* Open a Linux console terminal then go to your project directory and type this:
 +
<pre class="host">
 +
$ cd ~/tmp/apf6_the_full_howto
 +
$ find . -name "*_p0_pin_map.tcl" | xargs sed -iolt "/_mem_if_ddr3_emif_0_p0_get_input_clk_id {/a    return \$pll_output_node_id"
 +
</pre>
 +
{{Note| This is the [[DDR3-CycloneV_interface_description#The_DDR3_clock_hack | DDR3 clock hack]], each time synthesis is relaunched, this shell line must be relaunch}}
 +
* Right click on ''Assembler (Generate programming files)'' then '''Start'''
 +
* Convert bitstream to rbf with menu:
 +
<pre class="config">
 +
File
 +
  Convert Programming Files...
 +
</pre>
 +
* set these options:
 +
** Programming file type: '''Raw Binary File (.rbf)'''
 +
** Change the name to : '''output_files/tfht.rbf'''
 +
** Select '''SOF Data''' in '''Input files to convert''' then click on '''Add File...'''
 +
** add the file '''apf6_the_full_howto.sof''' in directory ''output_files''
 +
** checkbox '''Create CvP files (Generate tfht.periph.rbf and tfht.core.rbf)'''
 +
** click on '''Generate'''
 +
* These files should be present on directory '''output_files''':
 +
<pre class="host">
 +
ls -l tmp/apf6_the_full_howto/output_files/*.rbf
 +
tmp/apf6_the_full_howto/output_files/tfht.core.rbf
 +
tmp/apf6_the_full_howto/output_files/tfht.periph.rbf
 +
</pre>
 +
 
 +
{{Note| These bitstreams can be found under armadeus distribution in directory `firmware/tfht/bitstreams` }}
 +
 
 +
= Linux BSP configuration =
 +
 
 +
{{Note| This is not an Armadeus BSP tutorial, the BSP is supposed to be installed and compiled according to this [[Toolchain| explains]]}}
 +
 
 +
* Get a compiled BSP with following configuration : '''apf6_defconfig'''
 +
* Add [[Pci_debug| PCI debug]] package with '''make menuconfig'''
 +
<pre class="config">
 +
Target packages  --->
 +
    Hardware handling  --->
 +
        [*] pcidebug
 +
</pre>
 +
{{Note | On current Armadeus BSP, pcidebug is already installed by default}}
 +
* Then make the binaries and [[Target_Software_Installation |flash it]] on the apf6_sp.
 +
* Adding the rbf file to your [[Communicate_with_your_board_from_a_Linux_Host_(Basics)#TFTP_server | tftpboot]] directory.
 +
<pre class="host">
 +
$ cp -v output_files/tfht.*.rbf /tftpboot/
 +
‘output_files/tfht.core.rbf’ -> ‘/tftpboot/tfht.core.rbf’
 +
‘output_files/tfht.periph.rbf’ -> ‘/tftpboot/tfht.periph.rbf’
 +
</pre>
 +
 
 +
{{Note| Refer to [[PCIe FPGA loading]] to understand FPGA configuration with PCIe}}
 +
* In U-Boot download the periph.rbf and configure FPGA (don't forget to configure your tftp server ip):
 +
<pre class="apf">
 +
BIOS> tftpboot ${loadaddr} 192.168.0.214:tfht.periph.rbf
 +
BIOS> fpga load 0 ${loadaddr} ${filesize}
 +
</pre>
 +
* Boot Linux
 +
<pre class="apf">
 +
BIOS> boot
 +
...
 +
apf6 login: root
 +
</pre>
 +
 
 +
* The three BARx should be seen with '''lspci''' command :
 +
<pre class="apf">
 +
# lspci -v -s 01:00.0
 +
01:00.0 Unclassified device [0013]: Altera Corporation Device e001 (rev 01)
 +
        Flags: fast devsel, IRQ 341
 +
        Memory at 01800000 (32-bit, non-prefetchable) [disabled] [size=32K]
 +
        Memory at 01000000 (32-bit, non-prefetchable) [disabled] [size=8M]
 +
        Memory at 01808000 (32-bit, non-prefetchable) [disabled] [size=32]
 +
        Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+
 +
        Capabilities: [78] Power Management version 3
 +
        Capabilities: [80] Express Endpoint, MSI 00
 +
        Capabilities: [100] Virtual Channel
 +
        Capabilities: [200] Vendor Specific Information: ID=1172 Rev=0 Len=044 <?>
 +
</pre>
 +
* Configure network:
 +
<pre class="apf">
 +
$ udhcpc
 +
</pre>
 +
* Download core:
 +
<pre class="apf">
 +
$ tftp -g -r tfht.core.rbf 192.168.0.214
 +
</pre>
 +
* Load the core bitstream:
 +
<pre class="apf">
 +
$ load_fpga tfht.core.rbf
 +
Altera CvP 0000:01:00.0: enabling device (0140 -> 0142)
 +
Altera CvP 0000:01:00.0: Found and enabled PCI device with VID 0x1172, DID 0xE001
 +
successfully init Altera CVP with major 248
 +
Altera CvP 0000:01:00.0: Now starting CvP...
 +
Altera CvP 0000:01:00.0: CvP successful, application layer now ready
 +
8212+1 records in
 +
8212+1 records out
 +
</pre>
 +
 
 +
== User access to FPGA ==
 +
 
 +
=== DDR3 ===
 +
 
 +
* DDR3 is accessible via the BAR1, connect to BAR1:
 +
<pre class="apf">
 +
$ pci_debug -s 01:00.0 -b1
 +
</pre>
 +
 
 +
* Read first values of DDR3:
 +
<pre class="apf">
 +
PCI> d32 0 20
 +
 
 +
00000000: AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA
 +
00000010: 55555555 55555555 55555555 55555555
 +
</pre>
 +
 
 +
* Write values, then read back:
 +
<pre class="apf">
 +
PCI> c32 0 CAFEDECA
 +
PCI> c32 8 CACABEBE
 +
PCI> d32 0 20
 +
 
 +
00000000: CAFEDECA AAAAAAAA CACABEBE AAAAAAAA
 +
00000010: 55555555 55555555 55555555 55555555
 +
</pre>
 +
 
 +
* Exit pci_debug
 +
<pre class="apf">
 +
PCI> quit
 +
</pre>
 +
 
 +
Only first 8MB (0x800000) are accessible for the moment, to access next 8MB, the span extender must be configured on BAR0.
 +
 
 +
* Access BAR0 and configure [https://documentation.altera.com/#/00030683-AA$NT00064780 span extender]:
 +
<pre class="apf">
 +
$ pci_debug -s 01:00.0 -b0
 +
</pre>
 +
 
 +
* Read the span extender windows value at address 0x4000:
 +
<pre class="apf">
 +
PCI> d32 4000 1
 +
 
 +
00004000: 00000000
 +
</pre>
 +
 
 +
* Write the new required value :
 +
<pre class="apf">
 +
PCI> c32 4000 800000
 +
PCI> d32 4000 1
 +
 
 +
00004000: 00800000
 +
</pre>
 +
 
 +
* Quit:
 +
<pre class="apf">
 +
PCI> quit
 +
</pre>
 +
* then reconnect pci_debug to change for bar1:
 +
<pre class="apf">
 +
$ pci_debug -s 01:00.0 -b1
 +
</pre>
 +
 
 +
* Read value :
 +
<pre class="apf">
 +
PCI> d32 0 20
 +
 
 +
00000000: 6B2171EA DFA5DA76 6CFB3F47 7AF9F526
 +
00000010: DBD85BAB 7F56CF7A 78FAD73B ECF997FE
 +
</pre>
 +
 
 +
* Write some values:
 +
<pre class="apf">
 +
PCI> c32 0 12345678
 +
PCI> c32 4 9ABCDEF0
 +
PCI> c32 10 12345678
 +
PCI> c32 14 9ABCDEF0
 +
PCI> d32 0 20
 +
 
 +
00000000: 12345678 9ABCDEF0 6CFB3F47 7AF9F526
 +
00000010: 12345678 9ABCDEF0 78FAD73B ECF997FE
 +
</pre>
 +
 
 +
* Etc...
 +
 
 +
=== PIO (Button and led) ===
 +
 
 +
As seen on the first schematic of this tutorial, the [https://documentation.altera.com/#/00081203-NT$NT00080322 PIO] is plugged on the BAR2 of the PCIe.
 +
 
 +
* Then first, connect to BAR2 with pci_debug:
 +
<pre class="apf">
 +
$ pci_debug -s 01:00.0 -b2
 +
</pre>
 +
 
 +
'''Button'''
 +
* Read all PIO registers:
 +
<pre class="apf">
 +
PCI> d32 0 8
 +
 
 +
00000000: 00000001 00000000
 +
</pre>
 +
 
 +
* Leave button pressed and read again:
 +
<pre class="apf">
 +
PCI> d32 0 8
 +
 
 +
00000000: 00000000 00000000
 +
</pre>
 +
 
 +
'''led'''
 +
* set PIO[1] (led) as output:
 +
<pre class="apf">
 +
PCI> c32 4 2
 +
</pre>
 +
 
 +
* Turn on led
 +
<pre class="apf">
 +
PCI> c32 0 2
 +
</pre>
 +
 
 +
* Turn off led:
 +
<pre class="apf">
 +
PCI> c32 0 0
 +
</pre>
 +
 
 +
= Conclusion =
 +
 
 +
This tutorial should give you the basics to use the CycloneV soldered on the APF6_SP module. Now you should know:
 +
 
 +
* PCIe:
 +
** How to instanciate the CycloneV hard IP
 +
** How to read/write data with it from Linux with simple access
 +
 
 +
* DDR3:
 +
** How to instanciate and configure the UniPHY hard IP
 +
** How to connect it to PCIe
 +
** How to read/write simple data
 +
 
 +
* Simple I/O:
 +
** How to instanciate a simple port for I/O under Qsys
 +
** How to drive it from Linux
 +
 
 +
Some subjects '''are not covered by''' this tutorial:
 +
 
 +
* Managing PCIe interrupts
 +
* Write a Linux driver
 +
* Increase data bandwidth with PCIe-DMA (up to 140MB/s)
 +
** From the FPGA to the i.MX6
 +
** From the i.MX6 to the FPGA
 +
* Create a custom Qsys component
 +
* Using Serializer/Deserializer
 +
* And more ...
 +
 
 +
 
 +
If you are interested in a specific subject, please contact [http://www.opossom.com/english/contact.html Armadeus System] they should do somethings for you.
 +
 
 +
= Links & Resources=
 +
 
 +
* [http://www.armadeus.com/_downloads/apf6_SP/hardware/apf6_the_full_howto.zip The full archive of quartus 15.1.1 project](zip)

Latest revision as of 14:56, 12 November 2019


Introduction

This is the full howto make an APF6_SP CycloneV example design which uses :

  • PCIe
  • DDR3
  • General Purpose Input/Outputs
    • LED
    • Button

This howto is designed for :

  • APF6D_SP-M1GE4G-4m-BW-C4M384: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips
  • Quartus with following versions:
    • Quartus 17.0.0 : Quartus 17 without update
    • Quartus 19.1 : Quartus Intel version
    • To install Quartus see this page.
The full design for this howto

Make the quartus project

  • Open the "New Project Wizard" in Quartus
  • Select a directory for the design, for example /home/your_name/workspace and a project name apf6_the_full_howto:
  • Then Choose Empty Project and don't add any file yet -> press "Next"
  • Select the part number : 5CGXFC4C7U19C8 -> press "Next" -> "Next" -> "Finish"
  • Right click on "Cyclone V -> Devices..." in project navigator and click on Device and Pin Options
  • in CvP Settings set this options:
    • Configuration via Protocol: Core initialization and update
  • then select "OK" two times

Make the Qsys project

  • Open Qsys/Platform designer:
    • With Quartus 17: Tools->Qsys
    • With Quartus 19.1: Tools->Platform designer
  • Delete the component clk_0 then save the Qsys project as : qsys_tfht.qsys
  • Click on Finish and generate the component with all options left by default
  • Under Quartus project add the Qsys project (don't forget to click on add after selected here).
How to add a file in project
  • Right click on qsys_tfht.qsys and set as Top-Level Entity
  • Relaunch Qsys by double-clicking on the file in "Project Navigator"

Adding the PCIe and CvP

  • In IP Catalog select the component:
Library
    Interface Protocols
        PCI Express
            Avalon-MM Cyclone V Hard IP for PCI Express 
            Avalon-MM Cyclone V Hard IP for PCI Express Intel FPGA IP   <- 19.1
  • "Add" it to design.
  • PCIe configuration:
    • Number of Lanes: x1
    • Reference clock frequency: 125Mhz
    • check Use 62.5 MHz application clock
    • check Enable configuration via the PCIe link
    • BAR0: 32-bit non-prefetchable memory
    • BAR1: 32-bit non-prefetchable memory
    • BAR2: 32-bit non-prefetchable memory
    • Vendor ID: 0x00001172
    • Device ID: 0x0000e001
    • Class Code: 0x00001300: there is a bug in quartus, the actual class code is here 13
  • click on "Finish"
  • Export these signals with name in bold (right click in "Hierarchy", then choose "Connections" and "Export as:..."). Exported signals will be accessible on top of module:
    • refclk : refclk
    • npor: npor
    • hip_serial: hip_serial
  • connect Rxm_BAR0 on Cra (always right click in "Hierarchy", then "Connections"...)
  • Add CvP component reconfig driver, with all default parameters:
Library
    Interface Protocols
        PCI Express
            Altera PCIe Reconfig Driver
            Altera PCIe Reconfig Driver Intel FPGA IP <- 19.1
  • Connect previous IP signals with:
    • coreclkout -> reconfig_xcvr_clk
    • coreclkout -> pld_clk
    • nreset_status -> reconfig_xcvr_rst
  • Add CvP component transceiver, with all default parameters:
Library
    Interface Protocols
        Transceiver PHY
            Transceiver Reconfiguration Controller
            Transceiver Reconfiguration Controller Intel FPGA IP <- 19.1
  • Connect previous PCIe IP signals with:
    • coreclkout -> mgmt_clk_clk
    • nreset_status -> mgmt_rst_reset
    • reconfig_driver.reconfig_mgmt -> reconfig_mgmt
    • reconfig_driver.reconfig_busy -> reconfig_busy
    • pci.reconfig_to_xcvr -> reconfig_to_xcvr
    • pci.reconfig_from_xcvr -> reconfig_from_xcvr
  • you should end up with the following wiring:
Wiring result after PCIe configured

Adding the DDR3

Note Note: If not indicated, parameters should be left to default one !


UniPHY controller

  • In the IP catalog, select:
Library
    Memory Interfaces and Controllers
        Memory Interfaces with UniPHY
            DDR3 SDRAM Controller with UniPHY
            DDR3 SDRAM Controller with UniPHY Intel FPGA IP <- 19.1
  • Select checkbox Enable Hard External Memory Interface
  • In tab PHY Settings configure:
    • Speed Grade: 8
    • Memory clock frequency: 375Mhz
    • PLL reference clock frequency: 62.5Mhz
    • Rate on Avalon-MM interface: Full
    • supply Voltage: 1.35V DDR3L
  • In tab Memory Parameters
    • In "Presets" (right of the window) select: JEDEC DDR3-1G6 2GB X8 and apply
    • Memory device speed grade: 400Mhz
    • Total interface width: 24
    • Row address width: 14
  • In tab Controller Settings
    • checkbox: Generate power-of-2 data bus widths for Qsys or SOPC Builder
    • checkbox: Enable Configuration and Status Register Interface (Internal JTAG)
    • checkbox: Enable Error Detection and Correction Logic
  • Click on Finish
  • Connect following signals:
    • coreclkout -> pll_ref_clk
    • coreclkout -> mp_cmd_clk_0
    • coreclkout -> mp_rfifo_clk_0
    • coreclkout -> mp_wfifo_clk_0
    • coreclkout -> csr_clk
    • nreset_status -> global_reset
    • nreset_status -> soft_reset
    • nreset_status -> mp_cmd_reset_n_0
    • nreset_status -> mp_rfifo_reset_n_0
    • nreset_status -> mp_wfifo_reset_n_0
    • nreset_status -> csr_reset_n

Span extender

DDR3 memory is to large for PCIe BAR domain. Then we have to use a span extender to adapt PCIe BAR domain to DDR3 domain.

  • Select span expander in IP Library:
Library
    Basic Functions
        Bridges and Adaptors
            Memory Mapped
                Address Span Extender
  • Configure it with these options:
    • Datapath Width: 64bits
    • Slave Word Address Width: 20bits
    • press "Finish"
  • Connect following PCIe IP signals to extender one:
    • coreclckout -> clock
    • nreset_status -> reset
    • Rxm_BAR0 -> cntl
      • Change base address of cntl to 0x0000_4000
    • Rxm_BAR1 -> windowed_slave
  • And following extender signals to DDR3 one:
    • expanded_master -> mem_if_ddr3_emif_0.avl_0

Adding the button and LED (PIO)

Documentation of the PIO component can be found here.

  • In IP Library select:
Library
    Processors and Peripherals
        Peripherals
            PIO (Parallel I/O)
            PIO (Parallel I/O) Intel FPGA IP <- 19.1
  • Configure it with these options:
    • Width: 2
    • Direction: Bidir
    • checkbox Synchronously capture
    • Edge Type: RISING
    • checkbox Generate IRQ
    • IRQ Type: EDGE
  • click on "Finish"
  • export pio.external_connection to pio
  • Connect following signals:
    • coreclkout -> pio_0.clk
    • nreset_status -> pio_0.reset
    • Rxm_BAR2 -> pio_0.s1
    • pio_0.irq -> Rxmirq

Qsys component schematic

The resulting schematics should give the figure below.

The full connections in qsys component

Now click on finish and generate the component with all default parameters kept.

Generate bitstream with Quartus

  • Make a first synthesis, in Task window, right click on Analysis & Synthesis then choose Start
Compile Design
    + Analysis & Synthesis
Note Note: If you have a synthesis error here with 19.1 be sure that you are using your perl distribution.

Pinout

  • PCIe
    • Outside Quartus, create a file named 'pcie_pinout.tcl and edit it with your favorite text editor.
    • Copy all lines from here.
    • Save your file, then add it to the project inside Quartus
    • Run the script with following menu, select it then run:
Tools
    TCL Scripts ...
  • DDR3
    • Create a file named ddr3_pinout.tcl and edit it.
    • Copy all lines from here (Pinout placement and Technology pinout).
    • Save then add the file to project
    • Run the script with following menu, select it then run:
Tools
    TCL Scripts ...
  • Button & Led
    • Open the pinplanner by double-click in tasks:
Compile Design
    Analysis & Synthesis
        I/O Assignment Analysis
            Pin Planner
    • For button pio_export[0] select Location column and set it to : PIN_L9
    • For led pio_export[1] select Location column and set it to : PIN_M10
    • Close the Pin Planner

Synthesis, Place & route

  • Right click on Analysis and Synthesis then Start
  • Open a Linux console terminal then go to your project directory and type this:
$ cd ~/tmp/apf6_the_full_howto
$ find . -name "*_p0_pin_map.tcl" | xargs sed -iolt "/_mem_if_ddr3_emif_0_p0_get_input_clk_id {/a     return \$pll_output_node_id"
Note Note: This is the DDR3 clock hack, each time synthesis is relaunched, this shell line must be relaunch
  • Right click on Assembler (Generate programming files) then Start
  • Convert bitstream to rbf with menu:
File
   Convert Programming Files...
  • set these options:
    • Programming file type: Raw Binary File (.rbf)
    • Change the name to : output_files/tfht.rbf
    • Select SOF Data in Input files to convert then click on Add File...
    • add the file apf6_the_full_howto.sof in directory output_files
    • checkbox Create CvP files (Generate tfht.periph.rbf and tfht.core.rbf)
    • click on Generate
  • These files should be present on directory output_files:
ls -l tmp/apf6_the_full_howto/output_files/*.rbf
tmp/apf6_the_full_howto/output_files/tfht.core.rbf
tmp/apf6_the_full_howto/output_files/tfht.periph.rbf
Note Note: These bitstreams can be found under armadeus distribution in directory `firmware/tfht/bitstreams`


Linux BSP configuration

Note Note: This is not an Armadeus BSP tutorial, the BSP is supposed to be installed and compiled according to this explains


  • Get a compiled BSP with following configuration : apf6_defconfig
  • Add PCI debug package with make menuconfig
Target packages  --->
    Hardware handling  --->
        [*] pcidebug
Note Note: On current Armadeus BSP, pcidebug is already installed by default
  • Then make the binaries and flash it on the apf6_sp.
  • Adding the rbf file to your tftpboot directory.
$ cp -v output_files/tfht.*.rbf /tftpboot/
‘output_files/tfht.core.rbf’ -> ‘/tftpboot/tfht.core.rbf’
‘output_files/tfht.periph.rbf’ -> ‘/tftpboot/tfht.periph.rbf’
Note Note: Refer to PCIe FPGA loading to understand FPGA configuration with PCIe
  • In U-Boot download the periph.rbf and configure FPGA (don't forget to configure your tftp server ip):
BIOS> tftpboot ${loadaddr} 192.168.0.214:tfht.periph.rbf
BIOS> fpga load 0 ${loadaddr} ${filesize}
  • Boot Linux
BIOS> boot
...
apf6 login: root
  • The three BARx should be seen with lspci command :
# lspci -v -s 01:00.0
01:00.0 Unclassified device [0013]: Altera Corporation Device e001 (rev 01)
        Flags: fast devsel, IRQ 341
        Memory at 01800000 (32-bit, non-prefetchable) [disabled] [size=32K]
        Memory at 01000000 (32-bit, non-prefetchable) [disabled] [size=8M]
        Memory at 01808000 (32-bit, non-prefetchable) [disabled] [size=32]
        Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+
        Capabilities: [78] Power Management version 3
        Capabilities: [80] Express Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [200] Vendor Specific Information: ID=1172 Rev=0 Len=044 <?>
  • Configure network:
$ udhcpc
  • Download core:
$ tftp -g -r tfht.core.rbf 192.168.0.214
  • Load the core bitstream:
$ load_fpga tfht.core.rbf
Altera CvP 0000:01:00.0: enabling device (0140 -> 0142)
Altera CvP 0000:01:00.0: Found and enabled PCI device with VID 0x1172, DID 0xE001
successfully init Altera CVP with major 248
Altera CvP 0000:01:00.0: Now starting CvP...
Altera CvP 0000:01:00.0: CvP successful, application layer now ready
8212+1 records in
8212+1 records out

User access to FPGA

DDR3

  • DDR3 is accessible via the BAR1, connect to BAR1:
$ pci_debug -s 01:00.0 -b1
  • Read first values of DDR3:
PCI> d32 0 20

00000000: AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA 
00000010: 55555555 55555555 55555555 55555555
  • Write values, then read back:
PCI> c32 0 CAFEDECA
PCI> c32 8 CACABEBE
PCI> d32 0 20

00000000: CAFEDECA AAAAAAAA CACABEBE AAAAAAAA 
00000010: 55555555 55555555 55555555 55555555
  • Exit pci_debug
PCI> quit

Only first 8MB (0x800000) are accessible for the moment, to access next 8MB, the span extender must be configured on BAR0. 

$ pci_debug -s 01:00.0 -b0
  • Read the span extender windows value at address 0x4000:
PCI> d32 4000 1

00004000: 00000000
  • Write the new required value :
PCI> c32 4000 800000
PCI> d32 4000 1

00004000: 00800000
  • Quit: 
PCI> quit
  • then reconnect pci_debug to change for bar1:
$ pci_debug -s 01:00.0 -b1
  • Read value :
PCI> d32 0 20

00000000: 6B2171EA DFA5DA76 6CFB3F47 7AF9F526 
00000010: DBD85BAB 7F56CF7A 78FAD73B ECF997FE
  • Write some values:
PCI> c32 0 12345678
PCI> c32 4 9ABCDEF0
PCI> c32 10 12345678
PCI> c32 14 9ABCDEF0
PCI> d32 0 20

00000000: 12345678 9ABCDEF0 6CFB3F47 7AF9F526 
00000010: 12345678 9ABCDEF0 78FAD73B ECF997FE
  • Etc...

PIO (Button and led)

As seen on the first schematic of this tutorial, the PIO is plugged on the BAR2 of the PCIe.

  • Then first, connect to BAR2 with pci_debug:
$ pci_debug -s 01:00.0 -b2

Button

  • Read all PIO registers:
PCI> d32 0 8 

00000000: 00000001 00000000
  • Leave button pressed and read again:
PCI> d32 0 8

00000000: 00000000 00000000

led

  • set PIO[1] (led) as output:
PCI> c32 4 2
  • Turn on led
PCI> c32 0 2
  • Turn off led:
PCI> c32 0 0

Conclusion

This tutorial should give you the basics to use the CycloneV soldered on the APF6_SP module. Now you should know:

  • PCIe:
    • How to instanciate the CycloneV hard IP
    • How to read/write data with it from Linux with simple access
  • DDR3:
    • How to instanciate and configure the UniPHY hard IP
    • How to connect it to PCIe
    • How to read/write simple data
  • Simple I/O:
    • How to instanciate a simple port for I/O under Qsys
    • How to drive it from Linux

Some subjects are not covered by this tutorial:

  • Managing PCIe interrupts
  • Write a Linux driver
  • Increase data bandwidth with PCIe-DMA (up to 140MB/s)
    • From the FPGA to the i.MX6
    • From the i.MX6 to the FPGA
  • Create a custom Qsys component
  • Using Serializer/Deserializer
  • And more ...


If you are interested in a specific subject, please contact Armadeus System they should do somethings for you.

Links & Resources

Retrieved from "http://armadeus.org/wiki/index.php?title=APF6_SP_The_full_howto&oldid=14854"