GPU Programming - Science topic

visit this link

Helal Uddin Mullah

MATLAB may launch a parallel pool automatically based on your selections. To enable this option, go to the Home tab's Environment group and click Parallel > Parallel Preferences, followed by Automatically build a parallel pool. Set your solver to parallel processing mode.

Make a row vector with values ranging from -15 to 15. Use the gpuArray function to move it to the GPU and build a gpuArray object. To work with gpuArray objects, use any MATLAB function that supports gpuArray. MATLAB uses the GPU to do computations automatically.

To start a parallel pool on the cluster, use the parpool function. When you do this, parallel features like parfor loops and parfeval execute on the cluster workers. If you utilize GPU-enabled functions on gpuArrays, the operations will be executed on the GPU of the cluster worker.

Dear Artémis Anest

Basically, as you correctly stated, you can run specific algorithms implemented in R packages on the GPU, but general R code doesn't.

Yet, R has some useful GPU libraries(https://cran.r-project.org/web/views/HighPerformanceComputing.html), even if it's not R that's directly running on GPU, but the C/Cuda code.

You could also try https://github.com/gpuRcore/gpuRcuda

- https://github.com/gpuRcore/gpuRcublas - https://www.h2o.ai/blog/h2o4gpu-now-available-in-r/ - depending on your needs.

Best wishes,

Francesco

Thank you @Kristoffer and @Antonio for your help. Also sorry for the late reply. I will follow your instructions and will let you know if I get positive results.

hi Mr shrivastav

did you get any help?

I have the same problem with lammps.

please help me if you can

Variable scope allows you to create new variables and to share already created ones while providing checks to not create or share by accident. For details, see the https://www.tensorflow.org/api_docs/python/tf/variable_scope

i think the best way to understand it is through tensorgraph debugger

Hi Hisham,

If you already have the serial code and you have already decided that the thesis framework is accelerating this specific code using OpenACC then the "hottest" topics could be "which adaptations of the original code/algorithm can be done to help OpenACC in improving the application's performance" and "how does your proposals work on different GPUs (if you have more than one available), i.e., how general they are".

Of course, a comparison with the corresponding CUDA implementation can also be interesting.

Good luck,

Eduardo

While there is no universal answer to your question about hybrid computation, more and more people consider that the directive approach (OpenACC / OpenMP) to be the nominal way to go for porting large legacy codes, mostly because OpenAcc / OpenMP directives can be added incrementally in the code.

An interesting emerging alternative, which is probably more adapted to new codes aiming both at performance and portability across architectures (x86_64 multicore, nvidia GPU, AMD GPU, ARM, ...) is to use a "smart" library like Kokkos (developped at Sandia NL) or RAJA (devel at Livermore NL) or others to implement shared memory parallelism. These libraries provides "generic" concepts to express parallel loops (for, reduce, and so on). These libraries focus on node-level parallelism, and need to be used in combination with MPI for large scale distributed computation.

Another major advantage of these libraries, when compared to the directive approach, is that, and this is especially true for Kokkos, is that they also provide "architecture-aware" data containers. E.g. Kokkos provides arrays (called Kokkos::View) for CPU/GPU/... which allows you to access data like this data(i,j,k), very intuitively, and simplify the memory management when dealing with GPUs. Note that Kokkos library is a core building piece of the large parallel linear algebra package Trilinos.

I think that portability and more precisely "performance portability" is a keep concept today, that is being able to develop codes that can run efficiently on multiple platforms, not just one.

You can read/see them in a number of image viewers such as ITK-SNAP, Slicer, ImageJ etc. Or, you can play with the data in python/matlab.

I confirm Sidi,

Indeed, i have created a docker image with cuda and qtcreator in my docker hub registry.

you can check and use this image : https://hub.docker.com/r/amine2733/qt-opencv-gpu/

I confirm that you use Keras (>2.0) for exploiting multiple GPUs. However, the three GPUs need to be from the same generation.

In your case, there is no problem for using the two GTX 1080 TI, but for the third one (Tesla K20), I'm not sure. It is a different card.

Please, if you test that, can you inform me if ti worked with the three.

Hope this helped.

Best regards,

Sidi,

I don't think so, this is a 1 GB main memory device, if you are thinking on exploring small things maybe you can do something. But if you are thinking on doing some real work then it won't be enough. Efehan's advice is definitively a good one, although expensive :)

Hi,

I think CUDA is not that easier than OpenCL. I have programmed both from early stages of them (2008). There is some extra boilerplate code (which you can write once and use in all your projects, like me!), but the kernels (the functions executed on GPU) seem very similar. You can use OpenCL on all GPUs (even nVidia ones), Intel and AMD CPUs, FPGA, etc. CUDA is only for nVidia GPUs. Performance-wise, they are very similar, though nVidia pushes towards CUDA.

There are several books on OpenCL now. See "OpenCL Programming by Example" or "OpenCL in Action". Also, take a look at AMD OpenCL university kit (free, link below). After you get familiar with the concept, read, cover-to-cover, OpenCL manual as well as AMD and nVidia optimization guides, so you can write efficient programs.

Hope this helps!

Also, forgot to comment on the rest of the components. Those are fine. The Samsung EVO drive is great. There are some SSD drives that have the M2 interface and that is significantly faster. You may consider adding a cheaper 7200 rpm but large additional drive for data storage...keep the OS on the SSD drive as any data that needs to be accessed quickly.

Since computation is concerned, I would recommend TensorFlow because it supports distributed computing and multiple GPUs.

Me too.

You can use gpu profiling which will tell the portion of the gpu execution time of some code. 

But if you means is the theoritical efficiency than you must analyze the primitive  operation of your algorithm. Please refer to big O notation and the parallel algorihm books if you want to analyze theoritically.

Hopes it help you.

You should not go for a GPU solution out of the blue, unless you have already made an analysis of the specifics of the problem and be SURE that it can be benefit from the GPU characteristics.

If you have access to the OpenACC compilers, the port from a normal CPU implementation to GPU is not very complicated.

Thanks for your responses. However I have already explored these options. The problem is that the pct in matlab is supported only beyond R2010a. and GPUmat only supports up till the kepler architecture.

I am looking at freewares to help me use matlab 2009 with a quadro K620

You have quite an interesting problem there Thorsten. First of all, you're absolutely right that mailboxing (=sample marking and checking) is not something you want to do in a parallel environment if you have so many samples to generate.

There are two approaches you could consider. One is rejection sampling: first you compute an upper bound on the image value (should be easy from your description). Then for every pixel you generate a random number and decide if to sample it or not. I'm not going to derive the formula, but based on a) the pixel value, b) the upper bound from the first step, c) number of pixels in the image and d) number of desired samples, you should be able to derive the probability of sampling every given location, so that overall, the expected number of samples in the image is what you want.

Second approach could be seen as halftoning (dithering) the image. Look for Floyd-Steinberg if you're not familiar with this. Point is, you can interpret your problem as approximating the grayscale 'distance' image by a binary function (where '1' naturally means placing a sample in a given pixel). This is then precisely what halftoning via error diffusion does. Note that this will also require you to normalize the image, but that's doable in real-time I think.

Both of these should work, although the first one is much better parralelizable. The second one on the other hand will lead to a better spatial distribution of samples (i.e., it'll avoid sample clustering). Admittedly neither will give you exactly the number of samples you specify, but the expected number of samples will correspond to the desired one, and at that number of samples the deviation will be quite tiny I expect. On the other hand, both of the methods guarantee you 0 or 1 sample per pixel.

I found a document from ANSYS which declare that it is impossible to do it! on page 17

Yes, there are...

Just a quick search on github and I've found 3 sat solver projects which use GPU.

For CUDA version you can check cuda-sat-solver:

For OpenCL you can check clsat and opancl-satsolver:

The framework is a hard decision.

You can either use CUDA or OpenCL.

CUDA is used by NVIDIA. Its advantage is that HPC environments use also NVIDIA GPU clusters.

OpenCL is an open standard, used by many companies. It should be the future. Some CPUs also support it.

A brief follow-up... I just found two relevant articles, related to the topic discussed above.

One is "Differences in Floating-Point Arithmetic Between Intel® Xeon® Processors and the Intel® Xeon Phi™ Coprocessor", found at https://software.intel.com/en-us/articles/differences-in-floating-point-arithmetic-between-intel-xeon-processors-and-the-intel-xeon.

The other one is "Consistency of Floating-Point Results using the Intel® Compiler or Why doesn’t my application always give the same answer?", at https://software.intel.com/sites/default/files/managed/9d/20/FP_Consistency_300715.pdf.

These may be of interest to anyone using both Intel® Xeon® processors and Xeon Phi™ coprocessors, or the Intel® Compiler (or even if just performing floating-point computations in general).

Aslo you can try OpenCL

You will probably want to also take a look at cudaFuncSetCacheConfig, documented here:

For shared memory use on the K20 you may want to configure this to "cudaFuncCachePreferShared".

People looking to understand the nuts and bolts of GROMACS performance and how to get the most out of GPUs should (1) use the most recent software, rather than 4.6.5 which is quite outdated at this point and (2) read the following papers:

The performance depends on the quality of the hardware and the balance between CPU and GPU workloads. One cannot necessarily slap on an arbitrary GPU and expect massive gains.

I have three arrays of 20170320 pixels and a bank of 14500 points in 7 classes. if the bank is smaller classification is less good.

thanks for your help.

i currently trying a faster method ... it takes time for results....

Generally, GPUs get through one instruction per cycle and then hide latency by switching threads. I think that the scatter/gather instructions in NVIDIA GPUs are one cycle if all the data is in the same cache line, otherwise it slows down a lot. There's a whole banking issue with the cache lines as well, which is complex. So, if you actually use the scatter/gather capability to actually scatter and gather, performance slows a *lot*.

Although you mention Gean4, you haven't really described which results you're trying to merge.  Such merging obviously may depend on physics, rather than the trivial advice to use gather/reduce.  Mostly, it's not really clear what you have done, and want to do.  My impression is that Gean4 has reasonable MPI support built in:

so are you already using G4MPImanager/G4MPIsession ?

No, that won't work as you intend.

There is a new compiler Sambamba

that parallelizes automatically, but

for CPUs, not for GPUs.

In GPU programming, a huge set of processors

executes the same function called the kernel.

The threads that are running are only

distinguished by coordinates (1, 2, or 3

dimensions of a grid), and these coordinates

can be used to identify the part of the

problem they have to work on.

See "Thread execution" and following chapters in

And "GPU kernels" (page 16) in

Regards,

Joachim

Cuda processors are SIMD (single instruction multiple data) architectures. Cuda processors are extremely efficient at computation on data that is not dependent on one another. Depending on the algorithm several dependencies may arise that give way to a break in parallelism.This break in parallelism usually takes place through conditionals.

You would need to be specific on the algorithm to get a more specific answer, since integration could benefit from reduction operations in CUDA, but again it depends on the algorithm.

The key to succeed in this kind of project is to properly partition the application, which blocks will be executed in the dual core ARM and which blocks will be executed in the FPGA fabric. You must know the formulas of your algorithm(s) and see if they show dependency or not with respect to time. If algorithm(s) do not show such a dependency, they can easily implemented in the FPGA using VHDL. If the algorithm show dependency, you must look for ways how you can convert a serial algorithm in a parallel version and take advantage of languages such as VHDL. Another option is to write high level language as C/C++ and use OpenCL implementation from Altera or Vivado HLS and see what hardware produce, but note both OpenCL and Vivado HLS have limitations, and not sure if the hardware those tools produce will be efficient. Finally, use the dual core ARM just as a control unit, to start/end  tasks in your algorithm.

GPU programs are significantly different from CPU programs. In general, programs running on a CPU cannot be executed on the GPU. Most likely you cannot just run your Matlab code on the GPU.

The easiest way is to use special functions from the Parallel Toolbox (might not be included in your Matlab version). These functions will then automatically be executed on the GPU. Only question is if your algorithm is based on these functions. In any case you need to rewrite your code for the GPU.

Usually, GPU code is written in a special language, CUDA or OpenCL. The previous solution is just a wrapper to this. Matlab supports GPU functions (called kernels) written in CUDA to be executed. This is also part of the Parallel Toolbox. Note that CUDA only supports nVidia graphic cards. I haven't read up on this, but it might be possible that the Parallel Toolbox in general only works with nVidia graphic cards.

For further information I'd like to refer to the attached links. (BTW these are the first hits on Google!)

Sorry Mikhail, I forgot to mention that the answer was about intra-warp divergence. In CUDA, there are two kinds of control (execution) path divergence: inter-warp and intra-warp.

I'll try to give an illustrative example. Imagine there are two squads of (say) 32 soldiers each, executing orders of one officer. The officer can give only one order to only one of the squads at the given moment (limitation of SIMD machines). Their task is to to prepare dinner and dig trenches. The first squad is "armed" with knives and potatoes, while the second is "armed" with shovels. The officer gives the order to the first squad to lift their knives. While they are lifting their knives, they can not do anything else, so the officer gives the order to the second squad to lift their shovels. By then, the first squad finished lifting their knives, so the officer gives them the order to lift the first potato. By then, the second squad finished lifting shovels, so they are issued the order to thrust the shovels into ground. By then, the first squad finished lifting potatoes, so the officer gives them the order to put the blade of their knives on the potatoes they are holding. The officer turns his attention to the second squad. Unfortunately, one of the soldiers hit a solid rock with his shovel, so the officer tells the other soldiers of the squad to wait (with their shovels thrust into ground), and addresses specifically the soldier who hit the rock to bend down and reach for the rock. The officer turns his attention to the first squad and tells them to rotate their potatoes by 20 degrees (he'd probably have to specify the axis of rotation and direction, but that is irrelevant here :) Then the officer turns his attention to the second squad, tells the soldier who hit the rock to grab the rock, and the other soldiers of the second squad to wait. Back to the first squad, the officer tells them to rotate their potatoes by 20 degrees (with the same remark as before). Back to the second squad, the officer tells the solder who grabbed the rock to throw the rock away (he'd probably have to warn the soldier to avoid hitting the rest of the squad with the rock, but we'll ignore that :) and the other soldiers of the squad to wait. Back to the first squad, another 20 degrees. Back to the second squad, the officer tells the soldier who just threw the rock away to thrust his shovel into ground and the other soldiers to wait. Back to first squad, another 20 degrees. Back to the second squad, the officer issues the order to the whole squad to lift their shovels (hopefully, with some soil on them) - because the problematic solder is now in the same state as the others from his squad.

I guess you figured out from this illustration that each squad is actually a warp, and a soldier is actually a thread. Warps compete for the commander's attention (concurrent execution), but within each warp threads execute their work in parallel. That is, in parallel - until at least one of them has to execute a different instruction from the others. In that case, some threads will have to wait until the others finish their work. There is loss in parallelism until their execution paths converge into a single path once again. As you could see, there is no loss in efficiency among two different warps (squads) even though the work they did was completely different. Put it simply, the work of the first squad was not influenced by the second squad and vice-versa. That is called inter-warp divergence. On the other hand, there was some loss in efficiency withing the second squad because one of the soldiers was unable to catch up with the rest of the squad - so the rest of the squad was forced to wait for him. Soldiers in that squad had to execute, at least partially, different orders. That is called intra-warp divergence and it causes some penalty. In the example I gave, the penalty was not great. Imagine that there was another soldier who hit a gold bar, and another one who hit an electric cable. The officer would probably not give the first one the order to throw the gold bar away, so other soldiers (including the one who hit the rock and the one who hit the electric cable) would have to wait for commander's attention. The worst possible situation would be when every single soldier within the squad hits something different with the shovel - different in a sense that it requires different kind of action. The commander would have to tell each one of them how to solve the problem, while the others (within the same squad) would have to wait.

I hope I managed to explain (in essence) what is intra-warp divergence :)

There is a project to speedup postgres with GPGPU, this is a link to a pretty good presentation about it: http://www.slideshare.net/kaigai/gpgpu-accelerates-postgresql

In order to solve large non linear systems in parallel with MPI, you can use PETSc, it is very efficient

Hey David. The answer to your question is no. Not many people are fan of the idea in the MPI forum. When you think of how difficult it is to orchestrate MPI_Cancel, you can easily get why the idea is a hard sell.

As you said, you can implement it yourself on top of MPI ... but then you have to watch out for things like spurious message matching; that is, messages that you thought were "dead" (but are not) and then the middleware matches them internally and; at application level you think that they are messages that are subsequent to the one you cancelled (just one issue). I worked on that and can point to references if interested.

The webpage needs to be retrieved by the CPU. It does not really make sense to make this parallel. The reason for this is that you need to request the webpage via your network card (and your internet connection). There is a speed-limit to this. You will always need to wait for the content first. Todays CPUs will be fast enough to handle this amount of incoming data (just speaking of receiving and not analyzing). It would make sense, though, to send several requests in parallel. The reason for this is the implementation of TCP/IP which will start each request at a low rate and then over time will increase the speed according to your bandwidth. When you are only requesting the text of a webpage you will never reach the full potential of your internet bandwidth.

Analyzing the content of a webpage could be done in parallel, though. You would need to do tests to figure out if this would make sense. If the webpages are not coming in fast enough then a CPU might be enough to do the computations until the next webpage arrives. Only if the CPU cannot handle analyzing incoming webpages fast enough it makes sense to explore GPGPU implementations.

Dear Ahmed, this is a very general question. I have used the CUDA-Matlab integration for solving several types of problems. For instance, I have used it for solving large and sparse linear systems. Furthermore, I have used it for variable selection application in the context of multivariate calibration. In both cases, it was possible to demonstrate that the parallel version is faster than sequential one.

As far as I know, the Parallel Computing Toolbox (PCT) plugin still does not provide support for setting the number of blocks and threads per block. This is automatically defined by PCT in runtime through an optimized way.

I suggest you to read some papers of mine that describe this issue. Moreover, this link may be useful for you: https://www.ll.mit.edu/HPEC/agendas/proc07/Day3/11_Fatica_Poster.pdf

Best wishes and good luck with your research!

Please read first few chapters of this document. surely you can get good knowledge about CUDA

I think this debate is quite similar to DirectX vs. OpenGL. The only difference is that DirectX is bound to a single operating system, whereas CUDA is not. Both DirectX and CUDA are proprietary solutions whereas OpenGL and OpenCL are open standards (actually, both managed by the Khronos Group).

The question is really about your goals. Do you want maximum performance? Then CUDA might be the best choice. Updates for CUDA occur quite often supporting the most recent features of NVIDIA GPUs. But, you have vendor lock-in and can only use NVIDIA cards. Looking at other discussions, AMD graphics cards can provide "more bang for the buck", especially when it is about double precision operations. NVIDIA graphic cards artificially reduce the bandwidth for double precision computations. So, if you want to use CUDA with double precision you need to buy TESLA compute cards. These are more expensive. In summary, depending on your problem and if maximum performance (i.e. every last percent) is of your concern (and money is not) then CUDA is the best choice.

If you don't buy a new graphic card once a year to get the best performance or you don't have the money for an expensive card, OpenCL (and maybe AMD) is a valid alternative. Also, if you don't spend too much time on optimization and don't learn every new language/GPU feature as soon as it comes out, there is not much of a performance difference between OpenCL and CUDA in general.

My choice is really simple: I've chosen OpenCL. The reason for this is that I'm writing commercial software which should work with the hardware the client already has. He should not be forced to buy an NVIDIA card. Furthermore, hardware as old as back to 2010/2012 should still be supported, which means that I cannot use modern hardware features. With these restrictions there is not much of a performance difference between CUDA and OpenCL. But, the good thing is that I don't have to write my code twice--once for the GPU and once for the CPU. OpenCL kernels will support the CPU as well. (The reason for this is that even if a client does not have an OpenCL-capable GPU the software should still work.)

Looking at the battle DirectX vs. OpenGL, OpenGL has a good chance to win in the long run as Valve is switching to OpenGL and starts bringing out their own operating system SteamOS. The reasons are portability and compatibility. For the same reasons OpenCL might win over CUDA in the long run. Not now, but maybe 10-20 years from now. OpenCL will not die because it is necessary for Mac OS and for AMD hardware. The question is if NVIDIA will fight for CUDA the same way as Microsoft does for DirectX. In my opinion the cost of DirectX for Microsoft is already too high.

Hello, Martin! I have used GPU for solving linear systems and for variable selection in multivariate calibration problems. In my profile you can have access to some of my papers.

Best regards!

Video usually is highly compressed (e.g. mpeg, avi) in order to save disc-space or transmission band-width. If you decompress it (a suitable codec is needed), it will be converted to raw frames, e.g. a time-series of normal images. Such uncompressed (=raw) images are 2D matrices of pixel-values, and are suitable to feed to the GPU.

It highly depends on the problem which method will be better. And then you also need to define what you actually mean by "better": Better convergence? Better speed? Sometimes a slightly better convergence is not important because of bad performance. Currently, in our simulation software we are using BiCGSTAB(2). But, we have several problems which we cannot simulate because the solver will not converge. We will try AMG methods for these kinds of problems. But probably it will be too slow in the general case.

I think you can't read directly from a video stream or webcam to GPU memory. You have to first read each frame normally and then upload each frame to GPU memory. Then, you can perform operations on the GPU.

GPUDirect RDMA is only usable on Linux systems

Thanks, Guilherme! I will research about them too.

In my opinion no. The biggest problem is getting data on and off the GPU.

(The CPU has that problem too, as it spends much of its time waiting on data

which is not in the registers or cache.)

The newest (and most expensive) NVIDIA card has only 12GB of on-board memory.

If you can parallelize the problem well for the GPU, and fit the entire problem within 12 GB, and your I/O requirements are not large, then the problem can

be done on a GPU very quickly.

Alternatively, if you can come up with a way to fill up the 12 GB of memory

compute with only that for several minutes, then move a different 12GB piece of

data in, and compute on that (the old memory overlay method) then you

can do well.

Does WRF have these capabilities? (I don't think so.)

From the experience I have of playing around with GPGPU, I would most definitely agree with all other responses saying that it depends on what your aim is.

Do you want to port an existing application or are you starting on a new project, or are you just wanting to experiment / prototype to ascertain potential benefits? Will the code always run on a single vendor's hardware?

For what it's worth, here are my few cents on the various possibilities

- cuda/opencl. Having used both, the kernel code (the instructions actually executed on the GPU) is basically the same, both in terms of functionality and in terms of coding style and syntax, and performance on a single GPU is much the same for all tests I have performed. Host-side code is more different, with opencl being perhaps a little more complicated, but in most cases it is "boiler-plate code" that can be copy-pasted, and its intricacies don't need to be mastered for a first foray into GPGPU programming.

OpenCL allows portability -albeit with some effort- across vendors, and even across widely differing types of hardware, both current and future (HSA-enabled hardware for example looks very promising). I have a CFD code written in OpenCL which runs well on NVIDIA & AMD GPUs, and also on cpus. This is not possible with cuda. Incidently, if performance/$ is a metric of interest, AMD hardware should absolutely not be discounted too quickly.

On the other hand, the one really interesting feature cuda currently has over opencl is for multi-GPU communication. Memory can be transferred between cards without being buffered in CPU memory, which can significantly speed up transfers, and simplify programming. Various implementations of Cuda-aware MPI exist, allowing high-performance MPI communications between cards. While things like this may appear in the future for OpenCL, to my knowledge none of the current implementations allow it.

- OpenACC-style directives; they allow existing fortran or C code to be adapted progressively while (usually) retaining the initial purely CPU codebase. In my experience however, the effort required to obtain good performance using directive in "real" CFD code is on par with, or at least not an order of magnitude smaller than that required to start again from scratch in GPGPU language.

While current working compilers are commercial, GCC is on its way to allowing OpenACC-accelerated code to be compiled.

- code translation tools such as par4all (the only one I have tested) seem promising; with some tuning after the translation, I was able to obtain good performance on reasonably realistic tests

- scripting languages interfaced with cuda/opencl: they are GREAT for prototyping/testing, and indeed more and more complete codes seem to use python as "glue" to call high-perfomance GPU-accelerated kernels

Python is also fun to use, and a great way to start with cuda/opencl without all the annoying host-side code.

Thank you very much, Marco! This is going to be very helpful!!! Best Regards!

In this situation I would prefer to use OpenCV instead of OpenGL.

OpenCV has modules through which you can leverage GPUs.

OpenCV has a fairly good functions available for edge detection.

For segmentation there is a function for using watershed algorithm.

these links might help you to write segmentation algorithms

Phase correlation works very fast and flawlessly on GPUs.

It would be better if you could choose nvidia instead of AMD due to extensive support of CUDA libraries for image rendering/implementation. If in Laptop, I recommend newly released maxwell mobile series 860M for light purpose uses

and quadro 4100M for workstation kind of things

I always recommend desktop graphics cards depending on budget , I strongly suggest to wait 2-3 months for release of Geforce 880 series with major architectural improvements along with CUDA6 support

Can you provide any sample code?

Thanks.

I think that first we need to understand what you mean by GPU programming. You can either use the GPU for computations or graphics. The first is called GPGPU computing (General Purpose GPU) and you would use OpenCL, CUDA or OpenACC for this. The latter is traditional graphics programming using OpenGL or DirectX (though DirectX is restricted to Windows).

If you want to dive into graphics programming I would recommend OpenGL over DirectX, mainly because of portability. OpenGL is also the standard in scientific research. The best part for computer graphics is that the coordinate system has the 'correct' orientation for common mathematical descriptions. If you want to learn OpenGL then it does not work without learning to write shaders using GLSL. Look for books and tutorials that directly start with using shaders since with OpenGL versions 3 and 4 the so-called fixed function pipeline has been deprecated.

If you want to get into using the GPU for computations then you have the choice between OpenCL, CUDA, and OpenACC. OpenCL and CUDA are direct competitors. CUDA is more wide-spread, slightly easier to use, but only restricted to NVIDIA hardware (and x86 processors if you are using the PGI compiler). OpenCL on the other hand is supported by NVIDIA graphics cards, AMD GPUs, CPUs and APUs, and Intel CPUs, some of their on chip graphics and their Xeon Phi. I've also read about support of ARM CPUs(mostly used in mobile devices). OpenACC is probably easier to start with, especially if you are familiar with OpenMP. But, it is only supported by few compilers. It uses OpenCL in the background, so it is also quite portable. Since it does not give you a lot fine-grained control it is usually a little bit slower than fully optimized OpenCL or CUDA code.

Let us know what area of GPU programming you are interested in and we can then give you a few additional pointers.

The first chapter of "Accelerating Matlab with GPU computing: A primer with examples" has several useful suggestions for sppeding-up matlab code without using GPUs, pre allocation of arrays and vectorization included but there are several other tricks that are useful to know and use (I didn't know that the order of double loops: lines then columns or columns then lines affects processing speed, for instance).

@Ivan

I cannot entirely agree with your statements. And there are certainly other sides to the medal in many of the points you make.

1) Two or three years ago OpenCL finally caught up to CUDA performance-wise. Probably, CUDA is still faster on NVIDIA hardware, but it is only a very tiny percentage. In comparison to AMD's gaming hardware NVIDIA looses big time for double precision. But, actually this is the question that has been asked initially. Futhermore, OpenCL allows for extensions which then allows to access special hardware features if necessary. In the end both languages are equally powerful and the implementation of the respective drivers for CUDA and OpenCL decide about the actual performance.

2) There are different tools and libraries for CUDA. Most people only know the CUDA libraries; and quite frankly, these are very popular. But, with a quick search on Google I found the following two links:

I don't know about books and guides for OpenCL. But, I prefer quality over quantity. And there are certainly good OpenCL books out there. Fewer books also makes it easier to choose one ;)

3) The question seems to be not directed towards supercomputers. And there are many computers with AMD hardware. Considering the point of performance per dollar AMD is a viable choice compared to NVIDIA, especially if it is about double precision computation.

4) OpenCL only has an initial overhead during the setup phase. Usually, this is mostly Copy&Paste to get it working in the first place. The actual implementation for just one specific type of hardware does not have any overhead compared to CUDA.

5) The question is not about writing portable code. (And CUDA is not portable at all, so this is not a fair point.) When choosing only one target platform OpenCL is as easy to tune as CUDA.

Coming back to the original question, AMD is a viable choice. It is true that OpenCL might be slightly slower on NVIDIA hardware. But then again you wouldn't be switching from CUDA to OpenCL if you would keep using NVIDIA hardware. The only of Ivan's points that is applicable to your situation is that there is quite an overhead during initialization. But, this point can be countered by the lower cost for higher performance. In the end, you can save a lot of time with every computation with a higher programming overhead for writing the initial setup.

Simulation work takes lots of computing, so in my view the first priority goes to Number of CUDA cores or processing cores, next the processing power of the GPU, that is in the GFLOPS (for single and double precision floating point numbers). GFLOPS is related to number of cores.

Next is the memory system (size and bandwidth).More memory means more capability of storing more data/info of the particles to be simulated, more memory bandwidth means faster transaction of the data between CPU-GPU or GPU-memory.

The Bus interface is also another factor. It determines the data flow rate between the CPU and GPU. If the data to be tranferred betweed the CPU and GPU is large, this factor should not be ignored. For example PCI-e bus.

Now talking about the cards available nowdays, the max CUDA core counts to be 2880 yielding 5300+ GFLOPS for singe precision and 1400+ GFLOPS for double precision calculations, 12GB GDDR5 memory giving 288 GB/s of theoretical bandwidth (as for Nvidia Tesla K40, and costs $5000+). On the other hand, Nvidia Geforce GTX TITAN Black Edition packs similar theoretical performance but at around $1200.

As some of the others have pointed out. The texture cache is a read only cache to the DRAM and you can request values between the integer points and the texture cache returns the appropriate linearly, bilinearly, or trilinearly interpolated value. The interpolation is done in hardware and in my experience is as fast as a single memory access; however the interpolation is performed using only 8 fractional bits (see section E.2 in the CUDA C programming guide), so the interpolation is a bit courser than you would achieve on the Xeon Phi.

While I know that the interpolation is performed in hardware, I am not aware of a detailed description of the hardware implementation.

Samsung published an implementation of OpenACC for GCC:

CUBLAS is located in the lib or lib64 directory of your CUDA installation. If you have chosen the default install location, that would probably be:

/usr/local/cuda/lib64/libcublas.so

Be sure to export the lib or lib64 directory to your LD_LIBRARY_PATH to ensure your binary can find it when running your application.

Hello,

I am solving my conseved FDTD equation using spectral methods. When I used single precision the average of the field was shifting by a few percentage this is why I always use the double precision. The problem comes fro the Fourier transforms. I am now using double precision all the time and I think the performance loos is only 2-3 times.

My opinion is that if you are doing a simple Euler algorithm and evaluate all de lapcians by f(i+1)+f(i-1)-2*f(i) the single precision will be enough.

GPU is for sure promising but I am not so convinced it is to that extend useful in the "real and complex" model setup. What I mean is if using ROMS or similar atmo model like WRF there are zillion of code lines and to benefit fruitfully from GPU (this is my opinion) one should invest time into re-coding i.e. OpenCL, OpenACC. There were some attempts to code only part of the expensive dynamics (in WRF micro physics if I recall correctly) but didn't get to far. More promising to me looks Intel Phi which is getting to the speed improvement without big re-coding effort, basically you need to recompile your model.

I would like to hear opinion of the ocean/atmo community about Intel Phi and "standard" Xeons, benchmark, speedup, bottlenecks etc.

Cheers,

Ivica

Cusp has diagnal matrix storage format:

Boston University lections:

Maybe it worth to use it with Thrust - i dont know, both have high level C++ interface.

Mario, I am very glad to hear you raising this question. 64-bit precision is being blindly used as a panacea in so many applications, when the input arguments have only three or four decimals of accuracy and the only accuracy needed in the outputs is also three or four decimals. But numerically-ignorant programmers invoke 15 decimals of accuracy throughout a calculation, just to be safe. YES, there is massive savings possible over using 15-decimal "scratch" variables for the entire calculation, which is usually done because no one has the time or expertise to think about rounding error any more. You can increase speed and accuracy at the same time as you decrease memory footprint, pressure on bandwidth, energy usage, and power consumption, just by right-sizing the precision and dynamic range to what you need.

Any possibility of quantifying the error in your output as a function of errors in inputs? It's usually not easy to figure out, but if you can, you may be astonished at the economy of storage that the knowledge presents to you.

Double-precision IEEE floats provide over 600 orders of magnitude; I suspect you need less than 10 orders of magnitude in your calculation. That excess dynamic range consumes an 11-bit exponent when you probably only need about 5 bits of exponent. If you know the largest value and smallest value that your algorithm will ever require, by all means scale the problem to fit that range and used fixed-point math to get more digits of fraction, since you really don't need to spend precious bits on an oversized exponent. Nor do you likely need 52 digits of fraction, as provided by IEEE doubles. There is probably a way to get excellent results, better than what you're getting, using 32-bit fixed point. If you can figure out how, you'll nearly double your speed because almost all current architectures are bandwidth-limited, not FLOPS-limited.

If you're attempting to use C++, it may be useful to try coding initially in C and use python scripts to handle some of the brute force calculations. I'm suggesting this because CUDA can be bulky and hard to handle, specifically when attempting to get the correct memory usage on the device and optimizing block sizes. pyCUDA is a library that i've had some success using to do CUDA based calculations without having to go through actually coding in CUDA. There is a tradeoff of speed since you are not coding exactly what you want, but it is helpful to see if your idea works on the GPU. Good Luck!

The answer to your question could be "yes" or "no", depending on how you define "high-level" language (and "widely").

Traditionally, HLS has been on a bit of an alchemic quest in trying to translate ordinary sequential programs in traditional sequential programming languages (such as C and Java) into circuits. In that narrow interpretation, I think the answer must be "no." I agree with Mario that that goal is wishful thinking. Even tools that advertise this capability typically bait-and-switch their users --- instead of translating C to gates, they really translate "well-prepped" (as Mario put it) C to gates, and "prepping" here can mean many things, such as selecting a small subset of C that is amenable to synthesis, or significantly annotating segments of code with information required to produce reasonable implementations.

On the other hand, if you interpret "high-level language" as one that might abstract from specific details of circuits (clocks, specialized architectural elements such as ALUs and memory etc.), while at the same time exposing features that would be relevant to the algorithm designer (most importantly parallelism), then I think the answer could well be "yes". It would mean shifting some of the focus of HLS research from ways of *extracting* parallelism from sequential code to economical ways of letting users *express* concurrency and build significant applications doing so.

The "real benefit" of moving "everything possible" to GPU is close to none. In order to get benefit out of accelerators such as GPUs one has to find computationally expensive (= calculation dominated) parts of the programm which can run independently and seperate them into so called kernels. These kernels are then executed by the GPU.

This process is not always possible and will in no programm be "everything possible" ;) Some programmes have very little computation and a lot of copying memory around. These applications are bandwidth limited (= data dominated) and will not perform well on external accelerators.

So like in any development process: Find out what the properties of your problem are and find a better way to do it.

And finally: Don't use GPUs just because everybody seems to do that nowadays

And really finally: If you want to use GPUs, do so using OpenACC instead of CUDA!

Optical flow estimation algorithm is best for moving camera applications.

You can use Netbeans with CUDA, it is so efficient. If you like, I can send you a tutorial for creating Netbeans projects using CUDA

Thanks Mostafa,

Yes, I have juste tested it.

Best regards,

Dear Sidi,

I think that video surveillance applications should be a good use case of motion traking using 4K videos.

"SLI Frame Rendering: Combines two identical NVIDIA Quadro PCI Express graphics cards with an SLI connector to transparently scale application performance on a single display by presenting them as a single graphics card to the operating system."Therefore, we can use SLI in conjunction with CUDA to have two identical cards on my machine (any 8800 or Quadro 5600 or 4600) and program 256 multiprocessors as though they were one GPU?

SLI and CUDA are orthogonal concepts. The first is for automatic distribution of rasterization, the second is for addressing direct execution of code on the GPU. CUDA is not used for rendering (on- or offscreen). That is when using CUDA you can simply list all available cards in the machine and directly submit code to execute. This code has nothing to do with shader code - it is C-like. So you have a lot more control of what happens where and when.No, in general you don't want to use SLI if you plan on using the GPUs for compute instead of pure graphics applications. You will be able to access both GPUs as discrete devices from within your CUDA program. Note that you will need to explicitly divide work between the GPUs.

I don't have an explanation for why SLI isn't desirable for compute applications,

yup Intel had provided SDK for Xeon PHI architecture hardware