Some days ago I found that someone was looking for a D3.js expert. To prove one is an expert one has to pass a test, and one of the tasks on this test is to create a 3D color picker with RGB for axes.

The cube faces cannot be filled with a bi-dimensional linear gradients because such gradients are not supported. So, you have to explicitly write a loop to add the pixels.
Using SVG to add the pixels is a bad idea: SVG is an XML language, and uses a DOM tree. Using SVG will use a lot of memory and will slow down your computer. Use a canvas instead. Drawing on a canvas is done by simple Javascript commands, that add lines and shapes.
Following is a little code snippet that fills a cube face:

    for (i=x1; i<=x2;i++){
      for (j=y1; j<=y2; j++){
        rgb_arr[d.rgb_variable[0]]=i;
        rgb_arr[d.rgb_variable[1]]=j;
        ctx.fillStyle=d3.rgb(rgb_arr[0],rgb_arr[1],rgb_arr[2]);
        ctx.fillRect(i,j,1,1);
      }
    }

Now, to get the color where the mouse points, first get the position using the mouse event’s offsetX and offsetY. These properties will hold the correct value even if the canvas is rotated.
Then you can get the RGBA values of the pixel using method getImageData of the canvas’ context. The method returns the data of a rectangle defined by 4 arguments: x,t,width and height.
Following is an example:

   canvas.on('click', function(evt){
     var ctx=d3.event.target.getContext('2d');
     var pixelData = ctx.getImageData(d3.event.offsetX,d3.event.offsetY,1,1).data;
     alert(pixelData);
  });