only extract information for robot control guidelines

Hi,

can someone please give me any tip on how to config an application where i take data from the video only for evaluating the visual scene and control a robot.

for example the sequence could be:

get data from webcam

rescale and smoothing on a part of the image(possibly by hand.. because i want to customize the process),

on the result apply sobel filter and store the result on a matrix A

movement detector and store the result on a matrix B

simple change detector and store the result on a matrix C

other control systems use this data..

I need the fastest way to take the image..

I've seen that i can grab an image or use a codec or filter..

what is faster?

what is easier?

thanks

[771 byte] By [dimitria] at [2007-9-25]
# 1

Not entirely sure what you are asking for, but http://www.codeproject.com/useritems/activity.asp includes some java jmf source code for a very fast frame grabber which also parses the raw image data and includes well defined motion detection as well as some routines for gradients. It may get you started. Good luck.

andyblea at 2007-7-15 > top of java,Security,Cryptography...
# 2

>I need the fastest way to take the image..

I've seen that i can grab an image or use a codec or filter..

You use a codec, for instance VideoForWindows (vfw) to grab a frame and convert it to a buffered image. Then you apply a filter. You can easily make a sobel filter with a convolve operator, ie

float horizontalSobelMatrix[] = {

1.0f, -0.0f, -1.0f,

2.0f, 0.0f, -2.0f,

1.0f, 0.0f, -1.0f};

I use this code to get frames from a webcam. You can adapt it for your own purposes.

import java.awt.*;

import javax.swing.*;

import javax.media.*;

import javax.media.format.*;

import javax.media.util.*;

import javax.media.control.*;

import javax.media.protocol.*;

import java.util.*;

import java.awt.Image;

import java.awt.image.*;

import java.io.File;

public class VideoForWindowsCamera

implements Camera {

public Player player = null;

public CaptureDeviceInfo deviceInfo = null;

public MediaLocator mediaLocator = null;

public Buffer buf = null;

public Image img = null;

public VideoFormat vf = null;

public BufferToImage btoi = null;

public FrameGrabbingControl frameGrabbingControl;

public boolean IS_READY = false;

public String DEFAULT_CAMERA_DESCRIPTION =

"vfw:Microsoft WDM Image Capture (Win32):0";

public static String TITLE = "camera";

static int GUI_WIDTH = 320;

static int GUI_HEIGHT = 260;

public VideoForWindowsCamera(boolean isVisible) {

initialize(isVisible);

IS_READY = true;

}

public void initialize(boolean setVisible) {

deviceInfo = CaptureDeviceManager.getDevice(DEFAULT_CAMERA_DESCRIPTION);

mediaLocator = deviceInfo.getLocator();

try {

player = Manager.createRealizedPlayer(mediaLocator);

player.start();

Component comp;

if ( (comp = player.getVisualComponent()) != null) {

if (setVisible) {

JFrame gui = getGuiInstance();

gui.setSize(new Dimension(GUI_WIDTH, GUI_HEIGHT));

//gui.setLocation(new Point(1024 - 320, 0));

gui.add(comp, BorderLayout.CENTER);

gui.setVisible(true);

}

}

}

catch (Exception ex) {

ex.printStackTrace();

}

}

public BufferedImage grabOneFrame() {

frameGrabbingControl = (FrameGrabbingControl)

player.getControl("javax.media.control.FrameGrabbingControl");

buf = frameGrabbingControl.grabFrame();

// Convert it to an image - see Java Developers Almanac for the procedure

btoi = new BufferToImage( (VideoFormat) buf.getFormat());

img = btoi.createImage(buf);

return Factory.toBufferedImage(img);

}

public Image grabOneFrame(boolean getAsImage){

frameGrabbingControl = (FrameGrabbingControl)

player.getControl("javax.media.control.FrameGrabbingControl");

buf = frameGrabbingControl.grabFrame();

// Convert it to an image

btoi = new BufferToImage( (VideoFormat) buf.getFormat());

img = btoi.createImage(buf);

return img;

}

public static JFrame getGuiInstance() {

JFrame frame = new JFrame(TITLE);

ImageIcon icon = new ImageIcon("resources" + File.separator + "yr_icon.png");

frame.setIconImage(icon.getImage()); //new

frame.setTitle(TITLE);

frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

Dimension d = Toolkit.getDefaultToolkit().getScreenSize();

int x = d.width - GUI_WIDTH;

int y = 0;

frame.setLocation(x,y);

return frame;

}

public static void main(String[] args) {

VideoForWindowsCamera camera = new VideoForWindowsCamera(true);

}

}

public interface Camera {

public BufferedImage grabOneFrame();

}

messengersa at 2007-7-15 > top of java,Security,Cryptography...
# 3

thanks very much for the code

my only doubt is if i put the varius filters inside the codec it can be faster or not?

so i will generate there the various processed images and access them from another thread that control the robot..

How do you see this solution?

Someone told me that the grab operation is very slow

ps. your code doesn't works..

the variable factory isn't defined

81.. return Factory.toBufferedImage(img);

dimitria at 2007-7-15 > top of java,Security,Cryptography...
# 4

>i put the varius filters inside the codec it can be faster or not?

Perhaps you could reframe this question. What exactly is the problem?

>your code doesn't works..

- the Image to BufferedImage block is from the java developers almanac, as I indicated in the comments in the code fragment I use a BufferedImage because it lends itself to quick filtering. You could modify the code to return an Image quite easily, but if you can`t here is the class

public class Factory {

public static final int RGB_ALPHA = 111;

public static final int RGB_RED = 112;

public static final int RGB_GREEN = 116;

public static final int RGB_BLUE = 117;

// This method returns true if the specified image has transparent pixels

/**

*

* @param image Image

* @return boolean

*/

public static boolean hasAlpha(Image image) {

// If buffered image, the color model is readily available

if (image instanceof BufferedImage) {

BufferedImage bimage = (BufferedImage) image;

return bimage.getColorModel().hasAlpha();

}

// Use a pixel grabber to retrieve the image's color model;

// grabbing a single pixel is usually sufficient

PixelGrabber pg = new PixelGrabber(image, 0, 0, 1, 1, false);

try {

pg.grabPixels();

}

catch (InterruptedException e) {

}

// Get the image's color model

ColorModel cm = pg.getColorModel();

return cm.hasAlpha();

}

// This method returns a buffered image with the contents of an image

/**

*

* @param image Image

* @return BufferedImage

*/

public static BufferedImage toBufferedImage(Image image) {

if (image instanceof BufferedImage) {

return (BufferedImage) image;

}

// This code ensures that all the pixels in the image are loaded

image = new ImageIcon(image).getImage();

// Determine if the image has transparent pixels; for this method's

// implementation, see e661 Determining If an Image Has Transparent Pixels

boolean hasAlpha = hasAlpha(image);

// Create a buffered image with a format that's compatible with the screen

BufferedImage bimage = null;

GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();

try {

// Determine the type of transparency of the new buffered image

int transparency = Transparency.OPAQUE;

if (hasAlpha) {

transparency = Transparency.BITMASK;

}

// Create the buffered image

GraphicsDevice gs = ge.getDefaultScreenDevice();

GraphicsConfiguration gc = gs.getDefaultConfiguration();

bimage = gc.createCompatibleImage(

image.getWidth(null), image.getHeight(null), transparency);

}

catch (HeadlessException e) {

// The system does not have a screen

}

if (bimage == null) {

// Create a buffered image using the default color model

int type = BufferedImage.TYPE_INT_RGB;

if (hasAlpha) {

type = BufferedImage.TYPE_INT_ARGB;

}

bimage = new BufferedImage(image.getWidth(null), image.getHeight(null),

type);

}

// Copy image to buffered image

Graphics g = bimage.createGraphics();

// Paint the image onto the buffered image

g.drawImage(image, 0, 0, null);

g.dispose();

return bimage;

}

}

>How do you see this solution?

Depends what you are trying to achieve. I have no problems grabbing and processing frames from a lores webcam to use in robot control. It is a simple procedure to compare one image with another if you overlay the image with a grid, evaluate the pixels in a grid cell to produce a value, and compare that value with previous values.

regards

messengersa at 2007-7-15 > top of java,Security,Cryptography...
# 5

One last question, you was very kind to answer all my previews..

Do you think that rendering the camera output online can be disabled, while grabbing images when needed?

I don't want to lose this cpu time, it can be small but why waste it?

and there is some method to recycle memory to store the image grabed from the cam?

dimitria at 2007-7-15 > top of java,Security,Cryptography...
# 6

>Do you think that rendering the camera output online can be disabled, while grabbing images when needed?

You don`t have to display the camera output if you don`t want to.

You can close the input steam from the camera anytime, but it takes a (unpredictable) while to initialize the camera and input stream again. Obviously, your program thread will block until the camera initializes.

>I don't want to lose this cpu time, it can be small but why waste it?

What is your anxiety here?

>and there is some method to recycle memory to store the image grabed from the cam?

You can store the image in a Collection or an array of type Image[] or BufferedImage[].

regards

messengersa at 2007-7-15 > top of java,Security,Cryptography...