GPU Computing

Libraries | Languages | Compilers | APIs 

Language/API/Compilers for GPU

CUDA   (c/c++) (nvidia -> nvcc):-It is a programming language for Cuda architecture licensed to Nvidia .It is compiled by nvcc compler of NVIDIA.

OpenCL:-It is a free source language for Cuda architecture.It is platform independent.

Intel cilk plus

C++ AMP (Microsoft)

Direct Compute (Microsoft)

OpenACC

OpenHMPP

PGI CUDA (c/c++)

PGI Accelarator (C)

OpenGL

WebCL

WebGL

API/compilers for CUDA with other languages:

PGI CUDA (Fortran)

PGI Accelarator (Fortran)

CUDA Fortran (Portland group)

PyCUDA (Python)

alea.cuBase (.net + CUDA)

CUDA enabled libraries:

CUDA Math Library

CUFFT

CUSP

CUBLAS:-It is a linear algebra library used for algebra operations used in cuda programming.It is equalant to  BLAS which is used for linear algebra operations in normal C/C++ programming.

          Thrust:-Thrust is library in-built in cuda language .It is used to pass CPU vector to the GPU using a      raw -pointer.

CULA:-Cula is third party tool which is used for linear algebra library operations like SVD,SV in Cuda programming .It is equalant to LAPACK library which used in C/C++ programming for the same opearations.

CURAND:-It is used for the random number generation in Cuda programming and equalant to rand function of normal C/C++ programming .

CUSPARSE

MAGMA

          FLAME 

Jacket for MATLAB

ArrayFire

NPP (Nvidia Performance Primitives)

IMSL Fortran Numerical Library

GPU Debuggers

cugdb:-cugdb capable of handling thousands of threads running simultaneously on each GPU in the system.  CUDA-GDB delivers a seamless debugging experience that allows you to debug both the CPU and GPU portions of your application simultaneously.

          Nvidia Nsight:-is the ultimate development platform for heterogeneous computing. Work with powerful           debugging and profiling tools that enable you to fully optimize the performance of the CPU and   GPU.
 Not only do these feature-rich tools optimize performance, they help you gain a better understanding of your                  code - identify and analyze bottlenecks and observe the behavior of all system activities.

Total View

Performance analysis

nVidia NSIGHT

nVidia Visual Profiler

Compilers/Debuggers/Profilers ((C/C++/Fortran/MPI))

This is a list of Compilers, debuggers, and profilers for single core CPU, multi-core CPU and clusters.

======================================================

1) Single Core CPU: C/C++/Fortran

2) Multi Core CPU:      

               OpenMP (omp.h),

                   PTHREAD  (POSIX Threads)

                   Intel Parallel Building blocks ( PBB),

                          Intel Thread building blocks (TBB),

                          Intel Array building blocks (ABB)

                          Intel Cilk Plus 

3) Cluster (Multi-node): MPI

-------------------------------------------------------

OpenMP is provided by several vendors: Intel, Microsoft, IBM, GNU, etc.

VS has openMP by default. (omp.h)

=========================================

GNU (C/ C++ / Fortran)

C/C++:  gcc,cc,g++,c++,gcc-c++,cpp

Fortran:  g77,f90,f77,f95,gfortran,g95,gcc-gfortran

Profiler:  gprof

Debugger: gdb

Intel: (C/C++/Fortran/MPI)

C/C++: icc, icpc

Fortran: ifort

MPI: impi,  mpicc, mpiicpc, mpiifort

Debugger: idb

Profiler: iprof

Intel parallel studio , Intel cluster studio,Intel C++ Studio,Intel Fortran Studio

Intel parallel inspector,Intel Parallel Advisor,Intel parallel Amplifier,

Intel Parallel composer,Intel C++ composer,Intel Fortran composer

Intel VTUNE Amplifier,Intel Inspector,Intel Trace Analyzer and collector (itac)

Intel Parallel Building blocks ( PBB):  Intel Thread building blocks (TBB),Intel Array building blocks (ABB), Intel cilk+

Others:

PGI: pgicc, pgdbg, pgprof, pgdb, pgprof

NAG:  nagfor

IBM: XL fort, XL c++

Microsoft:msmpi, c,c++

===========================================================

Fortran:

ifort, nagfor, g77,f90,f77,f95,gfortran,g95, pgfort

C/C++:

gcc,cc,g++,c++,gcc-c++,cpp,    icc, icpc, mpicc, mpiicpc,      pgicc,    XL C/C++

MPI:

Intel (mpiicc, mpiicpc, mpiifort),Microsoft (msmpi), PGI (pgimpi), MPICH, MPICH2, MVAPICH, MVAPICH2, OPENMPI, mpif90

===========================================================

Compilers:

GNU: gcc,cc,g++,c++,gcc-c++,cpp,   g77,f90,f77,f95,gfortran,g95,gcc-gfortran

Intel:  icc, icpc, ifort,  impi,  mpicc, mpiicpc, mpiifort

PGI:  pgicc, pgifort, pgcuda

NAG: Nagfor

IBM: XL C++, XL Fortran

Microsoft: C/C++, MPI (msmpi)

Profilers:

GNU: gprof

Intel:  Intel VTune apmplifier, Intel trace Analyzer and collector (ITAC)

PGI (pgiprof)

Oprofile

Debuggers:

GNU (GDB)

Intel (IDB)

PGI (PGDBG)

--