Freigeben über


Wenming's Big Data and Big Compute Blog

All about running HPC and Big Data workload on the Microsoft Windows Azure platform. I have been and will continue to cover a variety of topics on Hadoop, HPC, Migration from / Interops with Unix, Tips and Tricks for running HPC and Big Data.

Data Science in a Box using IPython: Scipy and Scikit-Learn (3/4)

In the first two blogs of this series, we installed the IPython notebook using the minimum...

Date: 04/17/2013

Data Science in a Box using IPython: Installing IPython notebook (2/4)

In the previous blog, we demonstrated how to create a Windows Azure Linux VM in detail. We will...

Date: 04/09/2013

Data Science in a Box using IPython: Creating a Linux VM on Windows Azure (1/4)

I just returned from the Python in Finance Conference in New York, I would like to thank Bank of...

Date: 04/09/2013

Enter the Big Data Matrix: analyzing meanings and relations of everything (2/2)

Running the Python example step by step: We explained the basic idea behind LSA or latent semantic...

Date: 04/07/2013

New Breakthrough in Big Data Technologies: the NullSQL Paradigm shift

Mammoth the NullSQL tool Most of us by now understand the properties of big data. Many of us are...

Date: 04/01/2013

Make another small step, with the JavaScript Console Pig in HDInsight

Our previous blog, MapReduce on 27,000 books using multiple storage accounts and HDInsight showed...

Date: 03/31/2013

MapReduce on 27,000 books using multiple storage accounts and HDInsight

In our previous blog, Preparing and uploading datasets for HDInsight, we showed you some of the...

Date: 03/30/2013

Preparing and uploading datasets for HDInsight

In the previous blog ...

Date: 03/30/2013

Finding and pre-processing datasets for use with HDInsight

  Free datasets There are many difficult aspects associated with Big Data, getting a good,...

Date: 03/30/2013

Let there be Windows Azure HDInsight

Windows Azure HDInsight Service, formerly known as Hadoop on Windows Azure, is now available inside...

Date: 03/19/2013

Windows HPC Pack 2012 Tutorial (1/5): Installing Server 2012 on an IaaS VM

In a previous blog, I explained Big Compute and the newly released HPC Pack 2012 from...

Date: 12/19/2012

Windows HPC Pack 2012 released, HPC/Big Compute explained for Windows Azure developers

While most of you that have been using the Microsoft HPC Server product know the history and...

Date: 12/19/2012

Running Weather Research Forecast as a Service on Windows Azure

About 9 months ago, I deployed a Weather forecast demo at an internal Microsoft event, Techfest. The...

Date: 11/26/2012

Super computing 2012 and new Windows Azure HPC Hardware Announcement

The Super Computing conference attracts some of biggest names in the industry, academia, and...

Date: 11/21/2012

Hadoop On WindowsAzure Updated

HadoopOnAzure allows a user to run Hadoop on Microsoft Windows Azure as a service. It is currently...

Date: 08/22/2012

Deploying an HPC Cluster using just PowerShell (Part I)

Last month my colleague Don Pattee at the HPC team announced the availability of the Windows Azure...

Date: 12/06/2011

Just Published: Five learning samples for Windows HPC with Burst to Windows Azure covering Parametric Sweep, Cluster SOA, and Excel UDF offloading Programming Models

  If you have CPU intensive jobs that require 100s of Cores on Azure or on HPC on-premise...

Date: 05/06/2011

Windows HPC2008R2 SP2 Azure Burst Beta Documentation Available

  I will update here with more information as they become available for this exciting...

Date: 05/03/2011

Windows HPC Server 2008 R2 SP2 Beta is out!

I will be covering some of the new, and more complete Azure burst features on my blog, please stay...

Date: 05/02/2011

Don’t trust the weatherman? Running my own weather forecast on Windows HPC Server (Rain is coming back to Seattle Monday)

One of the interesting things living in Boulder Colorado for almost 20 years is that I bump into...

Date: 05/01/2011

Having a BLAST on Windows HPC with Windows Azure burst (1/3)

  My previous blog showed you how to make a movie using ray tracing software and the powerful...

Date: 05/01/2011

Got a powerful GPU? Run my World Wide Telescope 3D Seismic Simulation Tour

  For the last two years, I’ve been using my 6 core 3.3 ghz Intel Nehalem X 980 processor...

Date: 04/30/2011

HPC Server SP1: Burst to Azure with little or no programming

  We all know that Scientists and Engineers run Computationally intensive jobs. Many of us...

Date: 04/30/2011

Notes on Chemistry codes

I attended and gave a tutorial at the 11th LCI International conference last year at the Pittsburgh...

Date: 04/30/2011

Windows HPC Server 2008 R2 Launch Event in New York City

Windows HPC Server 2008 R2 or version 3 of the Windows HPC Server product has RTMed in July. The...

Date: 09/11/2010

New version: HPC Debugger Enhancements for MPI and Cluster-SOA

This package includes two enhancements to ease debugging HPC applications. The first feature...

Date: 01/13/2010

Windows HPC Server and the game of Go: Challenging 10^170 search space.

In the past year, the Microsoft HPC team has demonstrated the ManyFacesOfGo application running on...

Date: 01/06/2010

Windows HPC Server 2008R2 Beta1 Channel 9 Videos

At SC09 Microsoft Introduced Windows HPC Server 2008R2 Beta1.  Channel 9 videos for Beta1...

Date: 12/16/2009

Unix to Windows API dictionary project

As an HPC cross platform developer, I have spent a good part of my 20s working on scientific codes...

Date: 11/14/2009

Super computing 2009 next week

Quite a few folks from the Microsoft HPC team will be heading down to Portland next week to attend...

Date: 11/14/2009

Microsoft buys Interactive Supercomputing (Star-P maker)

Kyril Faenov, the General Manager, HPC and Parallel Computing Technologies, announced last week that...

Date: 09/28/2009

Installing SUA--- a Walk through

I have explained what SUA is and how it differs from Cygwin in previous blogs. Basically, SUA stands...

Date: 09/28/2009

5 minute tutorial on the amazingly easy to use MPI Cluster Debugger

The MPI Cluster Debugger (with C/C++ project templates) is now available via the Microsoft Download...

Date: 07/18/2009

Examples of linking in MSMPI library.

MSMPI is Microsoft’s implementation of MPI (Message passing interface) library.  I’ve explained...

Date: 06/26/2009

LIZARD released into the HPC wild

    One of my previous blog explains what linpack is:   LAPACK, LINPACK, ATLAS,...

Date: 06/26/2009

Windows HPC resource Kit version 2 available

resourcekit.windowshpc.net Finally version 2 is updated and online. Changes include: New Parallel...

Date: 05/01/2009

Linux Cluster International Conference

The 10th LCI (Linux Cluster Institute) international Conference on High Performance Clustered...

Date: 03/21/2009

HPC MATH ATTACKS: BLAS/LAPACK, LINPACK, ATLAS, DGEMM, ACML, MKL, CUDA

When it comes to HPC development, linear algebra libraries are the essential building blocks. ...

Date: 02/24/2009

mpiexec not working on windows XP? We got a patch.

Overview This update fixes an issue where the HPC Pack 2008 SDK was attempting to use a function...

Date: 02/13/2009

HPC Web cast Marathon into 2009!

High Performance Computing (HPC) Clusters: Reduce Complexity with IBM and Microsoft Please join us...

Date: 01/02/2009

We are HPCs @ PDC

I am at Microsoft PDC this week, demonstrating the Microsoft HPCS Server 2008. Please come by the...

Date: 10/27/2008

HPC Resource Kit

Yes, it's ready! I have compiled a 70 item kit full of white papers, presentations, videos, and...

Date: 10/23/2008

No scientist left behind with CRAY Supercomputer running Windows HPC Server 2008

Many people are still wondering exactly what Windows HPC Server 2008 is. On the launch day of the...

Date: 09/22/2008

Running PETSC on windows

PETSc, stands for portable, extensible toolkit for scientific computation, it is a suite of data...

Date: 09/08/2008

Porting your HPC Code to Windows

One of the most frequent questions I get when I talk to customers is, what is my best option for...

Date: 09/06/2008

HPC Basic Profile Videos

HPC Basic Profile Web Service or the HPCBP for short, is a web service, built using the Windows...

Date: 08/24/2008

Next>