This site contains OpenCL notes, tutorials, benchmarks, news.

Saturday, June 1, 2013

Tutorial: Simple start with OpenCL and C++

To begin programming in OpenCL is always hard. Let's try with the basic example. We want to sum two arrays together.

At first you need to install the OpenCL libraries and other files. AMD has for CPU's and their GPU's  AMD APP: http://developer.amd.com/tools-and-sdks/heterogeneous-computing/amd-accelerated-parallel-processing-app-sdk/downloads/. Intel has their OpenCL libraries at http://software.intel.com/en-us/vcsource/tools/opencl-sdk. And Nvidia has everything at https://developer.nvidia.com/cuda-downloads. In some cases the graphic drivers already include all the files you need. I recommend that you continue with the next step and if anything will go wrong return to this step and install the needed OpenCL SDK toolkits.



We will program in C++11. To ease everything we will use OpenCL C++ binding 1.1 from www.khronos.org/registry/cl/api/1.1/cl.hpp . manual for this binding is available at www.khronos.org/registry/cl/specs/opencl-cplusplus-1.1.pdf. It might happen that cl.hpp is already installed at your computer. If not then simply download C++ binding to folder of your project. Don't forget to turn on the C++11. In case of QtCreator add next line into the .pro file:
  QMAKE_CXXFLAGS += -std=c++0x
Also don't forget to use OpenCL library. In case of QtCreator add next line into the .pro file:
  LIBS+= -lOpenCL  
If you get any errors you need to adjust system variable to point to folder of OpenCL installation. You can also manually set path to OpenCL library path:
  LIBS+= -Lpath_to_openCL_libraries
Or you can simply write hard-coded path to OpenCL library:
  LIBS+=/usr/.../libOpenCL.so 
Let's start with coding. We will create simple console program which will use OpenCL to sum two arrays like C=A+B. For our simple sample we will need only two headers:
#include <iostream>
#include <CL/cl.hpp>
Everything else will happen inside main function. At start we need to get one of the OpenCL platforms. This is actually a driver you had previously installed. So platform can be from Nvidia, Intel, AMD....
int main(){
    //get all platforms (drivers)
    std::vector<cl::Platform> all_platforms;
    cl::Platform::get(&all_platforms);
    if(all_platforms.size()==0){
        std::cout<<" No platforms found. Check OpenCL installation!\n";
        exit(1);
    }
    cl::Platform default_platform=all_platforms[0];
    std::cout << "Using platform: "<<default_platform.getInfo<CL_PLATFORM_NAME>()<<"\n";

Once we selected the first platform (default_platform) we will use it in the next steps. Now we need to get device of our platform. For example AMD's platform has support for multiple devices (CPU's and GPU's). We will now select the first device (default_device):
    //get default device of the default platform
    std::vector<cl::Device> all_devices;
    default_platform.getDevices(CL_DEVICE_TYPE_ALL, &all_devices);
    if(all_devices.size()==0){
        std::cout<<" No devices found. Check OpenCL installation!\n";
        exit(1);
    }
    cl::Device default_device=all_devices[0];
    std::cout<< "Using device: "<<default_device.getInfo<CL_DEVICE_NAME>()<<"\n";

Now we need to create a Context. Imagine the Context as the runtime link to the our device and platform:
    cl::Context context({default_device});

Next we need to create the program which we want to execute on our device:
    cl::Program::Sources sources;

Actual source of our program(kernel) is there:
    // kernel calculates for each element C=A+B
    std::string kernel_code=
            "   void kernel simple_add(global const int* A, global const int* B, global int* C){       "
            "       C[get_global_id(0)]=A[get_global_id(0)]+B[get_global_id(0)];                 "
            "   }                                                                               ";                                                                          ";
This code simply calculates C=A+B. As we want that one thread calculates sum of only one element, we use get_global_id(0). get_global_id(0) means get id of current thread. Id's can go from 0 to get_global_size(0) - 1. get_global_size(0) means number of threads. What is 0? 0 means first dimension. OpenCL supports running kernels on 1D, 2D and 3D problems. We will use 1D array! This means 1D problem. 

Next we need our kernel sources to build. We also check for the errors at building:
    sources.push_back({kernel_code.c_str(),kernel_code.length()});

    cl::Program program(context,sources);
    if(program.build({default_device})!=CL_SUCCESS){
        std::cout<<" Error building: "<<program.getBuildInfo<CL_PROGRAM_BUILD_LOG>(default_device)<<"\n";
        exit(1);
    }
For arrays A, B, C we need to allocate the space on the device:
    // create buffers on the device
    cl::Buffer buffer_A(context,CL_MEM_READ_WRITE,sizeof(int)*10);
    cl::Buffer buffer_B(context,CL_MEM_READ_WRITE,sizeof(int)*10);
    cl::Buffer buffer_C(context,CL_MEM_READ_WRITE,sizeof(int)*10);
Arrays will have 10 element. We want to calculate sum of next arrays (A, B).
    int A[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
    int B[] = {0, 1, 2, 0, 1, 2, 0, 1, 2, 0};
We need to copy arrays from A and B to the device. This means that we will copy arrays from the host to the device. Host represents our main. At first we need to create a queue which is the queue to the commands we will send to the our device:
    //create queue to which we will push commands for the device.
    cl::CommandQueue queue(context,default_device);
Now we can copy data from arrays A and B to buffer_A and buffer_B which represent memory on the device:
    //write arrays A and B to the device
    queue.enqueueWriteBuffer(buffer_A,CL_TRUE,0,sizeof(int)*10,A);
    queue.enqueueWriteBuffer(buffer_B,CL_TRUE,0,sizeof(int)*10,B);
Now we can run the kernel which in parallel sums A and B and writes to C. We do this with KernelFunctor which runs the kernel on the device. Take a look at the "simple_add" this is the name of our kernel we wrote before. You can see the number 10. This corresponds to number of threads we want to run (our array size is 10):

    cl::KernelFunctor simple_add(cl::Kernel(program,"simple_add"),queue,cl::NullRange,cl::NDRange(10),cl::NullRange);
Here we actually set the arguments to kernel simple_add and run the kernel:
    simple_add(buffer_A, buffer_B, buffer_C);
At the end we want to print memory C on our device. At first we need to transfer data from the device to our program (host):
    int C[10];
    //read result C from the device to array C
    queue.enqueueReadBuffer(buffer_C,CL_TRUE,0,sizeof(int)*10,C);

    std::cout<<" result: \n";
    for(int i=0;i<10;i++){
        std::cout<<C[i]<<" ";
    }

    return 0;
}

This is it. Complete code is there:


#include <iostream>
#include <CL/cl.hpp>

int main(){
    //get all platforms (drivers)
    std::vector<cl::Platform> all_platforms;
    cl::Platform::get(&all_platforms);
    if(all_platforms.size()==0){
        std::cout<<" No platforms found. Check OpenCL installation!\n";
        exit(1);
    }
    cl::Platform default_platform=all_platforms[0];
    std::cout << "Using platform: "<<default_platform.getInfo<CL_PLATFORM_NAME>()<<"\n";

    //get default device of the default platform
    std::vector<cl::Device> all_devices;
    default_platform.getDevices(CL_DEVICE_TYPE_ALL, &all_devices);
    if(all_devices.size()==0){
        std::cout<<" No devices found. Check OpenCL installation!\n";
        exit(1);
    }
    cl::Device default_device=all_devices[0];
    std::cout<< "Using device: "<<default_device.getInfo<CL_DEVICE_NAME>()<<"\n";


    cl::Context context({default_device});

    cl::Program::Sources sources;

    // kernel calculates for each element C=A+B
    std::string kernel_code=
            "   void kernel simple_add(global const int* A, global const int* B, global int* C){       "
            "       C[get_global_id(0)]=A[get_global_id(0)]+B[get_global_id(0)];                 "
            "   }                                                                               ";
    sources.push_back({kernel_code.c_str(),kernel_code.length()});

    cl::Program program(context,sources);
    if(program.build({default_device})!=CL_SUCCESS){
        std::cout<<" Error building: "<<program.getBuildInfo<CL_PROGRAM_BUILD_LOG>(default_device)<<"\n";
        exit(1);
    }


    // create buffers on the device
    cl::Buffer buffer_A(context,CL_MEM_READ_WRITE,sizeof(int)*10);
    cl::Buffer buffer_B(context,CL_MEM_READ_WRITE,sizeof(int)*10);
    cl::Buffer buffer_C(context,CL_MEM_READ_WRITE,sizeof(int)*10);

    int A[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
    int B[] = {0, 1, 2, 0, 1, 2, 0, 1, 2, 0};

    //create queue to which we will push commands for the device.
    cl::CommandQueue queue(context,default_device);

    //write arrays A and B to the device
    queue.enqueueWriteBuffer(buffer_A,CL_TRUE,0,sizeof(int)*10,A);
    queue.enqueueWriteBuffer(buffer_B,CL_TRUE,0,sizeof(int)*10,B);


    //run the kernel
    cl::KernelFunctor simple_add(cl::Kernel(program,"simple_add"),queue,cl::NullRange,cl::NDRange(10),cl::NullRange);
    simple_add(buffer_A,buffer_B,buffer_C);

    //alternative way to run the kernel
    /*cl::Kernel kernel_add=cl::Kernel(program,"simple_add");
    kernel_add.setArg(0,buffer_A);
    kernel_add.setArg(1,buffer_B);
    kernel_add.setArg(2,buffer_C);
    queue.enqueueNDRangeKernel(kernel_add,cl::NullRange,cl::NDRange(10),cl::NullRange);
    queue.finish();*/

    int C[10];
    //read result C from the device to array C
    queue.enqueueReadBuffer(buffer_C,CL_TRUE,0,sizeof(int)*10,C);

    std::cout<<" result: \n";
    for(int i=0;i<10;i++){
        std::cout<<C[i]<<" ";
    }

    return 0;
}

333 comments:

  1. Hi,

    thanks for the tutorial.
    To make it work with OpenCL 1.2 and corresponding cl.hpp:

    //run the kernel
    //cl 1.1
    //cl::KernelFunctor simple_add(cl::Kernel(program,"simple_add"),queue,cl::NullRange,cl::NDRange(10),cl::NullRange);
    //simple_add(buffer_A,buffer_B,buffer_C);

    //cl 1.2
    cl::make_kernel simple_add(cl::Kernel(program,"simple_add"));
    cl::EnqueueArgs eargs(queue,cl::NullRange,cl::NDRange(10),cl::NullRange);
    simple_add(eargs, buffer_A,buffer_B,buffer_C).wait();

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. cl::make_kernel needs template arguments, but I think anything between angle brackets gets eaten, which is probably why stéphane's code came out looking wrong. In this example, they need to be cl::Buffer& repeated three times (two input buffers, one output buffer).

      Sorry, too lazy to figure out the mark-up for embedded code.

      Delete
    3. //run the kernel
      auto simple_add = cl::make_kernel(program, "simple_add");
      cl::EnqueueArgs eargs(queue,cl::NullRange,cl::NDRange(10),cl::NullRange);
      simple_add(eargs, buffer_A,buffer_B,buffer_C).wait();

      Overwrite line 61-62 with this for opencl 1.2 works.

      Delete
    4. Thanks a lot!! This was extremely helpful

      Delete
    5. Hi, I use opencl 2.1 and I replaced line 61-62 with:

      cl::make_kernel simple_add(cl::Kernel(program, "simple_add"));
      cl::EnqueueArgs eargs(queue, cl::NullRange, cl::NDRange(10), cl::NullRange);
      simple_add(eargs, buffer_A, buffer_B, buffer_C).wait();

      Note that different from cl1.2, the cl::make_kernel is a class template, so there follows "".

      Delete
    6. For OpenCL 1.2, use the "Alternative Way" to run the kernel given in the complete code example of the original tutorial:

      cl::Kernel kernel_add=cl::Kernel(program,"simple_add"); kernel_add.setArg(0,buffer_A);
      kernel_add.setArg(1,buffer_B);
      kernel_add.setArg(2,buffer_C);

      queue.enqueueNDRangeKernel(kernel_add,cl::NullRange,cl::NDRange(10),cl::NullRange);

      queue.finish();

      Also, define the macro CL_USE_DEPRECATED_OPENCL_1_2_APIS at the beginning of your code:

      #define CL_USE_DEPRECATED_OPENCL_1_2_APIS


      (The other methods above require class template arguments)

      Delete
  2. Wow a very concise and easy to understand example. Thank you!

    ReplyDelete
  3. it is possible to bind C code or embedded C Code ..???

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Thank you for this tutorial. Unfortunately there is a problem using it with the current NVIDIA OpenCL ICD (the library that dispatches API calls to the appropriate driver), which is a missing function in the context of cl::Device. You can fix this by placing

    #define CL_USE_DEPRECATED_OPENCL_1_1_APIS
    #include iostream
    #include CL/cl.h
    #undef CL_VERSION_1_2
    #include CL/cl.hpp

    where filenames need to be embraced by < and >, at the very beginning.

    ReplyDelete
  6. Very helpful introduction to OpenCL, thank you!

    ReplyDelete
  7. Several errors occur when running this code. Where can be a problem? Thanks in advance.

    Error 1 error C2661: 'std::vector<_Ty>::push_back' : no overloaded function takes 2 arguments c:\xxxx\sumadd.cpp 34

    Error 2 error C2664: 'cl_int cl::Program::build(const std::vector<_Ty> &,const char *,void (__stdcall *)(cl_program,void *),void *) const' : cannot convert parameter 1 from 'cl::Device' to 'const std::vector<_Ty> &' c:\xxxx\sumadd.cpp 37

    3 IntelliSense: no instance of overloaded function "std::vector<_Ty, _Alloc>::push_back [with _Ty=std::pair, _Alloc=std::allocator>]" matches the argument list
    argument types are: (const char *, size_t)
    object type is: cl::Program::Sources c:\xxxx\SumAdd.cpp 34


    4 IntelliSense: no instance of overloaded function "cl::Program::build" matches the argument list
    argument types are: (cl::Device)
    object type is: cl::Program c:\xxxx\SumAdd.cpp 37

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. // add to the top
      #include < vector > (without the spaces)

      Delete
  8. Does OpenCL have any trait associated with it that would allow me to use a Macro to determine if OpenCL is available or not through my C++ code?
    Eg.
    #ifdef OPENCL

    ReplyDelete


  9. Thanks for sharing the very useful info about clanguage and please keep updating........

    ReplyDelete
  10. Great blog ! Keep writing !

    ReplyDelete
  11. Nice tutorial. Thanks for sharing the valuable info about c Training. it’s really helpful. Who want to learn c language this blog most helpful. Keep sharing on updated tutorials…..

    ReplyDelete
  12. I followed this example and I am now able to debug my open cl program! (in a crude way by writing values to a buffer... but it works!)

    thanks, great post

    ReplyDelete
  13. This is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article.
    iphone app development in hyderabad
    ios app training in bangalore
    ios app development in hyderabad

    ReplyDelete
  14. Thanks for sharing this useful information. Keep up the good work
    DevOps Online Training

    ReplyDelete
  15. Thanks you for sharing this unique useful information content with us.
    Datastage Online Training



    ReplyDelete
  16. It is wonderful. I was able to run the whole code with little tweeks with Visual Studio 2015. Thanks :)

    ReplyDelete
  17. I am obliged to you for sharing this piece of information here and updating us with your resourceful guidance. Hope this might benefit many learners. Keep sharing this gainful articles and continue updating us.
    AWS Training in Chennai
    DevOps Training in Chennai
    CCNA Course in Chennai
    RPA Training in Chennai
    Pyhton Training in Chennai
    R Programming Training in Chennai
    Angularjs Training in Chennai

    ReplyDelete
  18. Nice post. Thanks for sharing! I want people to know just how good this information is in your article. It’s interesting content and Great work.
    Thanks & Regards,
    VRIT Professionals,
    No.1 Leading Web Designing Training Institute In Chennai.

    And also those who are looking for
    Web Designing Training Institute in Chennai
    SEO Training Institute in Chennai
    Photoshop Training Institute in Chennai
    PHP & Mysql Training Institute in Chennai
    Android Training Institute in Chennai

    ReplyDelete
  19. This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information regarding Microsoft Azure which is latest and newest,

    Regards,
    Ramya

    azure training in chennai
    azure training center in chennai
    best azure training in chennai
    azure devops training in chenna
    azure training institute in chennai

    ReplyDelete
  20. Outstanding blog thanks for sharing such wonderful blog with us ,after long time came across such knowlegeble blog.
    R Training Institute in Chennai | R Programming Training in Chennai

    ReplyDelete
  21. Nice Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    Check out : hadoop training in Chennai
    big data training in chennai
    big data hadoop training in chennai
    big data training and placement in chennai

    ReplyDelete
  22. One of the best content i have found on internet for Data Science training in Chennai .Every point for Data Science training in Chennai is explained in so detail,So its very easy to catch the content for Data Science training in Chennai .keep sharing more contents for Trending Technologies and also updating this content for Data Science and keep helping others.
    Cheers !
    Thanks and regards ,
    Data Science course in Velachery
    Data Scientists course in chennai
    Best Data Science course in chennai
    Top data science institute in chennai

    ReplyDelete
  23. I get an error when building:

    \main.cpp|38|error: no matching function for call to 'cl::Program::build()'|

    ReplyDelete
  24. I tried out and it seems to work. But it is not faster then calculating with CPU, even with 1000000 additions. What is wrong?

    Rolf

    ReplyDelete
  25. Thanks for this blog. It is more Interesting...
    C++ Techniques

    ReplyDelete
  26. This is a nice Site to watch out for and we provided information on
    vidmate make sure you can check it out and keep on visiting our Site.

    ReplyDelete
  27. I feel this article have given such a lot of vital info for me. And I am satisfied studying your article. However wanna commentary on few general things, The website style is ideal, the articles are truly nice.
    Interior Designers in Chennai | Interior Decorators in Chennai | Best Interior Designers in Chennai | Home Interior designers in Chennai | Modular Kitchen in Chennai

    ReplyDelete
  28. Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows.

    ReplyDelete
  29. Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows.

    ReplyDelete
  30. <a href="https://vidmate.vin/

    ReplyDelete
  31. Download latest audio and video file fromvidmate

    ReplyDelete


  32. Thanks for sharing the valuable information here. So i think i got some useful information with this content. Thank you and please keep update like this informative details.


    erp software development company in us
    erp software company in us
    list of erp software companies in us
    erp in chennai
    list of erp software companies in chennai
    Latest CRM Software

    ReplyDelete
  33. Hello, thenks for this example. Building success, but when I run this progrum ? I have an incorrect result, like result:
    -858993460 -858993460 -858993460 -858993460 -858993460 -858993460 -858993460 -858993460 -858993460 -858993460
    Help please, what the problem?

    ReplyDelete
  34. Thanks for sharing an informative blog keep rocking bring more details.I like the helpful info you provide in your articles. I’ll bookmark your weblog and check again here regularly. I am quite sure I will learn much new stuff right here! Good luck for the next!
    mobile application development training online
    mobile app development course
    mobile application development course
    learn mobile application development
    mobile app development training
    app development training
    mobile application development training
    mobile app development course online
    online mobile application development

    ReplyDelete
  35. Vanskeligheter( van bướm ) vil passere. På samme måte som( van điện từ ) regnet utenfor( van 1 chiều ) vinduet, hvor nostalgisk( van an toàn ) er det som til slutt( van bướm tay quay ) vil fjerne himmelen.

    ReplyDelete
  36. It is very useful information at my studies time, i really very impressed very well articles and worth information, i can remember more days that articles.

    catering services in chennai
    tasty catering services in chennai
    best catering services in chennai
    top catering services in chennai
    veg Catering services in chennai

    ReplyDelete
  37. You have provided a nice article, Thank you very much for this one. And I hope this will be useful for many people. And I am waiting for your next post keep on updating these kinds of knowledgeable things
    Android App Development Course in Chennai
    Android Development Course in Bangalore
    App Development Course in Coimbatore
    Mobile Application Development Course in Coimbatore


    ReplyDelete

  38. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    best microservices online training

    ReplyDelete
  39. Great share. Hands on basic make master in programming. Start with basic to job oriented training in Nagpur.
    -----------------------------------------------
    Webgurukul IT Training & Placement Institute

    ReplyDelete
  40. Hello Admin!

    Thanks for the post. It was very interesting and meaningful. I really appreciate it! Keep updating stuffs like this. If you are looking for the Advertising Agency in Chennai / Printing in Chennai , Visit us now..

    ReplyDelete
  41. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    best angular js online training

    ReplyDelete
  42. Useful information Thank-you for sharing. really helpful keep sharing your views. Please visit link mentioned below and share your views on them.
    best hvac training in lucknow
    job oriented training institute in lucknow
    best php training in lucknow
    digital marketing training in lucknow

    ReplyDelete
  43. Really i found this article more informative, thanks for sharing this article! Also Check here

    Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows

    Vidmate App Download

    Vidmate apk for Android devices

    Vidmate App

    download Vidmate for Windows PC

    download Vidmate for Windows PC Free

    Vidmate Download for Windows 10

    Download Vidmate for iOS

    Download Vidmate for Blackberry

    Vidmate For IOS and Blackberry OS

    ReplyDelete

  44. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    best workday studio online training

    ReplyDelete
  45. i found this article more informative, thanks for sharing this article!
    showbox
    showbox for pc

    ReplyDelete
  46. Awesome.....Really very informative and creative contents. This concept is a good way to enhance the knowledge. Thanks for sharing.
    Please do visit my website hope you'll like it: http://oceaninfosystem.com/
    Best software development company in Lucknow
    Best summer training company in Lucknow
    Best MLM company in Lucknow
    Best software development company
    Best summer internship institute Lucknow

    ReplyDelete
  47. Great Article. Thank-you for such useful information. I have read many of your post and this post is wonderful....
    http://pihmct.com/
    Best hotel management college
    Top hotel management college
    Best hotel management college near me
    diploma in hotel management
    Best hotel management institute
    Best institute for hotel management
    Get trained by professional and build your career in top Multi-national companies.

    ReplyDelete
  48. Great Article. Thank-you for such useful information. I have read many of your post and this post is wonderful....
    http://pihmct.com/
    Best hotel management college
    Top hotel management college
    Best hotel management college near me
    diploma in hotel management
    Best hotel management institute
    Best institute for hotel management
    Get trained by professional and build your career in top Multi-national companies.

    ReplyDelete
  49. Great Article. Thank you for sharing! Really an awesome post for every one.

    IEEE Final Year projects Project Centers in Chennai are consistently sought after. Final Year Students Projects take a shot at them to improve their aptitudes, while specialists like the enjoyment in interfering with innovation. For experts, it's an alternate ball game through and through. Smaller than expected IEEE Final Year project centers ground for all fragments of CSE & IT engineers hoping to assemble. Final Year Project Domains for IT It gives you tips and rules that is progressively critical to consider while choosing any final year project point.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Spring Framework Corporate TRaining the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai

    ReplyDelete
  50. Informative blog. Thanks for sharing, this post is wonderful....all of your posts are just awesome. Please do visit my website hope you'll like it:
    Best trust in Lucknow
    Best NGO in Lucknow
    Best social worker in Lucknow
    Social worker in Lucknow
    Best govt. health scheme in Lucknow
    NGO in Lucknow

    ReplyDelete
  51. Best tutorial blog share.
    Get job based , live project IT course training in Nagpur.
    Search more here: PHP Classes in Nagpur - Web Designing Classes in Nagpur

    ReplyDelete
  52. I find this blog to be very interesting and very resourceful. I would say that your blogs are really interesting and informative for me and this article explained everything in detail.
    Best pathology in lucknow
    Best pathology in jankipuram
    Diagnostic centre in lucknow
    X-ray pathology in jankipuram
    Best diagnostic centre in lucknow
    Pathology centre in jankipuram

    ReplyDelete
  53. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    angular js online training

    ReplyDelete

  54. Great sharing article, I like to visit your website
    free software download

    ReplyDelete
  55. GrueBleen – One of the best branding and marketing agency in Riyadh- Saudi Arabia. If you need the details, click the below link
    Branding Agency Riyadh
    Marketing Agency Riyadh

    ReplyDelete
  56. BSc Cardio Vascular Technology is one of the best demanding courses in recent times. Here you can check the all details of this course and the best college to study in Bangalore for this course. Just click the below mentioned link.
    BSc Cardiac Care Technology Colleges In Bangalore

    ReplyDelete
  57. Thanks for submit this post. Here I would like to share about the best college for Optometry. Here is the details of best BSc Optometry colleges in Bangalore. If you are looking to study BSc Optometry in Bangalore, click the below link.
    Optometry Colleges In Bangalore

    ReplyDelete
  58. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.
    data analytics courses
    data science interview questions

    ReplyDelete
  59. Both are really great. But its purely related to your taste. I prefer pursuing a degree in computer science and start learning digital marketing.
    ExcelR Digital Marketing Courses In Bangalore

    ReplyDelete
  60. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    best microservices online training

    ReplyDelete
  61. I am inspired with your post writing style & how continuously you describe this topic on software testing tutorial . After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.

    ReplyDelete
  62. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck... Thank you!!! digital marketing courses in Bangalore

    ReplyDelete
  63. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. bangalore digital marketing course

    ReplyDelete

  64. This post is really nice and informative. The explanation given is really comprehensive and informative. I also want to say about the complete digital marketing course

    ReplyDelete
  65. Awesome article, it was exceptionally helpful! I simply began in this and I'm becoming more acquainted with it better. The post is written in very a good manner and it contains many useful information for me. Thank you very much and will look for more postings from you.


    digital marketing blog
    digital marketing bloggers
    digital marketing blogs
    digital marketing blogs in india
    digital marketing blog 2020
    digital marketing blog sites
    skartec's digital marketing blog
    skartec's blog
    digital marketing course
    digital marketing course in chennai
    digital marketing training
    skartec digital marketing academy

    ReplyDelete
  66. i-LEND is an online marketplace connecting borrowers and lenders for loans. Although i-LEND verifies credentials of registered users on the site, it does not guarantee any loan offers by lenders nor does it guarantee any repayments by borrowers. Users make offers/loan requests at their own discretion with the understanding of the risks involved in such transactions including loss of entire capital and/or no guarantee of recovery. Please read our Legal agreements to understand more. borrow money

    ReplyDelete
  67. I have to search sites with relevant information on given topic ExcelR Digital Marketing Course Pune and provide them to teacher our opinion and the article.

    ReplyDelete
  68. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    servicenow online training
    best servicenow online training
    top servicenow online training

    ReplyDelete
  69. Are you looking for play bazaar were you get the latest satta result.

    ReplyDelete
  70. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science courses

    ReplyDelete
  71. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    welcome to akilmanati
    akilmanati

    ReplyDelete
  72. I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business.satta king

    ReplyDelete
  73. Second Innings Home is the first and only premium home & health care service in India. Second Innings Home proposed across the nation features a beautiful campus ideally located in a well-maintained gated community in the format of a Star Hotel with luxurious amenities. It’s convenient to enjoy the privacy and to be near the city and nearby facilities. And yet it retains a sense of community spirit and the warmth of a small community. good retirement homes in Hyderabad

    ReplyDelete
  74. Can I just say what a comfort to find an individual who truly knows what they are talking about on the net. You actually realize how to bring an issue to light and make works it important. More people should check this out and understand this side of your story. I was surprised that you're not more popular because you certainly possess the gift.

    ReplyDelete
  75. With the help of creative designing team TSS advertising company provides different branding and marketing strategies in advertising industry...
    https://www.tss-adv.com/branding-and-marketing

    ReplyDelete
  76. I grow to be greater than satisfied to find this fantastic website on line. I need to to thanks in your time because of this first-rate look at!! I truly cherished each bit of it and I have you ever ever bookmarked to look new facts for yourwebsite.

    ReplyDelete
  77. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    workday studio online training
    best workday studio online training
    top workday studio online training

    ReplyDelete
  78. You have provided a nice article, Thank you very much for this one. And I hope this will be useful for many people. And I am waiting for your next post keep on updating these kinds of knowledgeable things
    Android Training in Chennai
    Android Course in Chennai
    App Development Course in Chennai
    Android Development Course in Chennai
    Android App Development Course in Chennai

    ReplyDelete
  79. You have provided a nice article, Thank you very much for this one. And I hope this will be useful for many people. And I am waiting for your next post keep on updating these kinds of knowledgeable things
    App Development Course in Chennai
    Android Development Course in Chennai
    Android Training Institutes in Bangalore
    Android App Development Course in Bangalore
    Android Course in Coimbatore
    Android App Development Course in Coimbatore
    Android Course in Madurai

    ReplyDelete
  80. NAGAQQ | AGEN BANDARQ | BANDARQ ONLINE | ADUQ ONLINE | DOMINOQQ TERBAIK

    Yang Merupakan Agen Bandarq, Domino 99, Dan Bandar Poker Online Terpercaya di asia hadir untuk anda semua dengan permainan permainan menarik dan bonus menarik untuk anda semua

    Bonus yang diberikan NagaQQ :
    * Bonus rollingan 0.5%,setiap senin di bagikannya
    * Bonus Refferal 10% + 10%,seumur hidup
    * Bonus Jackpot, yang dapat anda dapatkan dengan mudah
    * Minimal Depo 15.000
    * Minimal WD 20.000

    Memegang Gelar atau title sebagai QQ Online Terbaik di masanya

    Games Yang di Hadirkan NagaQQ :
    * Poker Online
    * BandarQ
    * Domino99
    * Bandar Poker
    * Bandar66
    * Sakong
    * Capsa Susun
    * AduQ
    * Perang Bacarrat (New Game)

    Tersedia Deposit Via pulsa :
    Telkomsel & XL

    Info Lebih lanjut Kunjungi :
    Website : NAGAQQ
    Facebook : NagaQQ Official
    Kontakk : Info NagaQQ
    linktree : Agen Judi Online
    WHATSAPP : +855977509035
    Line : Cs_nagaQQ
    TELEGRAM : +855967014811


    BACA JUGA BLOGSPORT KAMI YANG LAIN:
    agen bandarq terbaik
    Winner NagaQQ
    Daftar NagaQQ
    Agen Poker Online

    ReplyDelete
  81. Great blog !It is best institute.Top Training institute In chennai
    http://chennaitraining.in/oracle-dba-training-in-chennai/
    http://chennaitraining.in/sql-server-dba-training-in-chennai/
    http://chennaitraining.in/teradata-training-in-chennai/
    http://chennaitraining.in/sap-hr-training-in-chennai/
    http://chennaitraining.in/sap-fico-training-in-chennai/
    http://chennaitraining.in/sap-abap-training-in-chennai/

    ReplyDelete
  82. TiketQQ Agen BandarQ Domino99 Bandar Poker dan Bandar66 Online Terbaik Di Asia

    Website paling ternama dan paling terpercaya di Asia ^^
    Sistem pelayanan 24 Jam Non-Stop bersama dengan CS Berpengalaman respon tercepat :)

    * Minimal DEPOSIT Rp 15.000,-
    * Minimal WITHDRAW Rp 15.000,-
    * Tersedia 9 game dalam 1 USER ID
    * Bonus TO 0,5% Setiap 5 hari
    * Bonus Refferal 20%
    * Proses Deposit & Withdraw PALING CEPAT
    * Sistem keamanan Terbaru & Terjamin
    * Poker Online Terpercaya
    * Live chat yang Responsive
    * Support lebih banyak bank LOKAL


    < Contact Us >
    Website : TiketQQ
    Facebook : TiketQQ
    Twitter : TiketQQ
    Instagram : TiketQQ
    LINE : Tiketqq
    WA : +855885063246
    Kontak : TiketQQ
    Blog : Cerita Dewasa

    Agen Judi Online Aman Dan Terpercaya TiketQQ

    ReplyDelete

  83. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    blockchain online training
    best blockchain online training
    top blockchain online training

    ReplyDelete
  84. It’s hard to come by experienced people about this subject, but you seem like you know what you’re talking about! Thanks.
    Java Training in Bangalore
    Python Training In Bangalore

    ReplyDelete
  85. article from a very amazing blog, Good Job, Thank you for presenting a wide variety of information that is very interesting to see in this artikle
    Home elevators Melbourne
    Home lifts

    ReplyDelete
  86. Awesome blog, I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the
    good work!.data analytics courses

    ReplyDelete

  87. TeaTv Apk Is Another One Of The Best Online Streaming Application For Smartphone Users. This Application Is Available For Android & IOS Users. Also, You Can Connect This Application Easily With Amazon Firestick, Android Smart TVs, Start Settop Box, Etc. There are more apk to online streaming is Mediabox apk for smartphone users ( Android and ios ) and
    Best Wishes Quotes 
    Watch Free online Movies 
    onhax 
    onhax android 
    Hulu Apk 

    ReplyDelete
  88. Your blog is splendid, I follow and read continuously the blogs that you share, they have some really important information. M glad to be in touch plz keep up the good work.
    Digital Marketing Courses in Pune

    ReplyDelete
  89. shabe barat
    Shab E Baraat When the 7th month of the

    Islamic calendar Rajab ends, the 8th month of the lunar calendar begins, this month is referred to as Shabaan.

    The importance, it is believed, lies with the 15th night of this month. shab e barat ki nafil namaz in

    hindi
    shab e barat ki namaz,

    shab e barat namaz, shab e barat namaz niyat, shab e barat namaz rakats, shab e barat quotes,
    shabe barat ki namaz, shabe barat namaz, shab e barat 2020shab e barat in india
    shab e barat dua, shab e barat nafil namaj


    --------------------------------------------------------------------

    Ramadan Kareem

    Wishes, Quotes, Greetings
     ? Then, this collection of Ramadan wishes and Ramadan Mubarak

    messages is for you. Here, you’ll find inspirational Ramadan Mubarak greetings that are hoping your

    recipient to have a blessed celebration of Ramadan. happy ramadan, happy ramadan wishes, happy ramzan,
    ramadan greetings, ramadan greetings in English, ramadankareem quotes, ramadan kareem wishes, ramadan wishes, ramzan quotes, ramzan. Ramadan calendar 2020, ramadan 2020 calendar, Ramadan mubarak 2020

    Thanks all

    ReplyDelete
  90. It’s hard to come by experienced people about this subject, but you seem like you know what you’re talking about! Thanks.
    Java Training in Bangalore
    Python Training In Bangalore

    ReplyDelete
  91. Kudos for delivering such an informative post. Have understood a lot from this concept. Gonna follow your updates from now on.

    Machine Learning Training In Hyderabad

    ReplyDelete
  92. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge. | Home lifts

    ReplyDelete
  93. Astrum InfoTech Agency is the Best Digital Marketing Company in Delhi, They help to generate the profit and visitors traffic on website. If you are looking for grow the online visibility of your business then please contact us. We Offer Best Digital Marketing Service like SEO Service, SMO Service, PPC Service, Facebook Marketing Services, Email marketing service, graphic design services, website designing service and website development service etc.

    ReplyDelete
  94. keep up the good work. this is an Ossam post. This is to helpful, i have read here all post. i am impressed. thank you. this is our Data Science course in Mumbai
    data science course in mumbai | https://www.excelr.com/data-science-course-training-in-mumbai

    ReplyDelete
  95. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.
    data analytics courses in mumbai

    ReplyDelete
  96. A very nice guide.it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    Machine Learning Training in Hyderabad

    ReplyDelete
  97. wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
    Data science Interview Questions
    Data Science Course

    ReplyDelete
  98. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science courses

    ReplyDelete