Maker Pro
Maker Pro

Keyboard / Mouse Input Device Design??

E

Eeyore

Jan 1, 1970
0
Peter said:
I need to do the MOUSE and the KEYBOARD. I have patented technology that can
intelligently recognize anything on any GUI screen. There are some applications
that get their keyboard input directly from the hardware. There might be some
applications that do the same thing for the mouse. I want a single solution that
always works on the Intel platform that can be used for automated testing on any
OS.

Uh ?

Can you translate that into engineerese ?

As in "do the MOUSE and the KEYBOARD" !!!!!!

Graham
 
E

Eeyore

Jan 1, 1970
0
Spehro said:
It took him longer than a more sane person because he
insisted on translating their C to assembler.

Good Lord !

Why didn't he use a dis-assembler ?

Graham
 
E

Eeyore

Jan 1, 1970
0
Peter said:
I need a way to feed keystrokes to applications that get their keystrokes
directly form the hardware.

What damn hardware ? You're very poor at explaining yourself.

I want the same sort of solution for the mouse.

What solution.

What are actually trying to make and don't go all coy about it being patented
either.

Graham
 
E

Eeyore

Jan 1, 1970
0
Peter said:
I am building a universal GUI testing system based on www.SeeScreen.com
technology that can always intelligently "see" anything on the screen by
recognizing graphical objects from their pixels.

What's that got to do with a mouse and keyboard.

Getting an explanation of what you're on about is like getting blood out of a
stone !

You want a 'box' that simulates keystrokes and mouse actions. That bit's easy.
What's going to *drive* it and how is *that* going to communicate with the 'box'
?

Graham
 
P

Peter Olcott

Jan 1, 1970
0
Eeyore said:
Uh ?

Can you translate that into engineerese ?

As in "do the MOUSE and the KEYBOARD" !!!!!!

Graham

I need a hardware device that can feed keystrokes to the keyboard port, and feed
mouse actions to the mouse port using software to provide this data. This is to
automatically test application programs as if it was a human operator typing
with the keyboard, or operating the mouse.
 
E

Eeyore

Jan 1, 1970
0
Peter said:
I need a hardware device that can feed keystrokes to the keyboard port, and feed
mouse actions to the mouse port using software to provide this data. This is to
automatically test application programs as if it was a human operator typing
with the keyboard, or operating the mouse.

And how is it going to receive this data ? Any limitations ?

Graham
 
J

Jan Panteltje

Jan 1, 1970
0
Are you saying that doing a screen capture is hard, or recognizing the graphical
objects from the screen capture? (I got the last part solved).

I am saying tha tto capture the screen in Linux, you must have access to the
display buffer.
If it runs Xwindows, in an application a buffer is created

A sequece would be something like this:
Display *mydisplay;
mydisplay = XOpenDisplay(displayname);
XDrawPoint(mydisplay, topwindow, xptestgc, x, y);
etc... (omitted important stuff).

'mydisplay' will be internal to the application, can even be a local variable...
So if your application needs to read say graphic text, or boxes from the
display, you have no access.
You cannot grab data from the graphics card either, as all are different
(bypass driver).
There are many modes in X too, and there is the framebuffer that can be used.
And then the same again for applications that use vgalib or just the
basic text console.
I would not know how to do it in a way that worked for _all_ systems known
on Linux.

This is assuming your application reads what is on the screen, what you
told me.
 
R

Rich Grise

Jan 1, 1970
0
I need a hardware device that can feed keystrokes to the keyboard port, and feed
mouse actions to the mouse port using software to provide this data. This is to
automatically test application programs as if it was a human operator typing
with the keyboard, or operating the mouse.

Well, here's some ideas on the keyboard.
http://www.abiengr.com/~sysop/KeyZilla/

I don't know how to do the mouse, but there's _got_ to be something out
there on the web; maybe google for "ps2 mouse interface" or something.

Good Luck!
Rich
 
L

Le Chaud Lapin

Jan 1, 1970
0
Peter said:
I am building a universal GUI testing system based on www.SeeScreen.com
technology that can always intelligently "see" anything on the screen by
recognizing graphical objects from their pixels.

This makes things clearer.

The hardware devices you speak of could be prototyped in perhaps a week
(for one device; the interfaces for mouse and keyboard are similar) and
made reliable in perhaps a little over a month using a PIC.

http://en.wikipedia.org/wiki/PIC_microcontroller

This book will have all you need to know about the various
keyboard/mouse interfaces. Note the $145 price tag (it's worth it):

http://www.amazon.com/Indispensable...ef=sr_1_1/102-6543581-0078538?ie=UTF8&s=books

But if you don't want to buy the book, look at the PIC page + these
links:

http://www.beyondlogic.org/keyboard/keybrd.htm
http://www.computer-engineering.org/ps2protocol/
http://www.electronic-engineering.ch/microchip/projects/keyboard/v1xx/keyboard_v1xx.html
http://www.simandl.cz/stranky/elektro/keyboard/keyboard_a.htm

On your web site (www.seescreen.com) you mentioned your solution could
be used for testing, but you did not say what would drive the devices
to generate their "key strokes" and "mouse movements". Naturally, you
would not want to hard-code the seqences in the device. The
inflexiblity would severely limit its usefulness. Whether you
hard-code or do a new burn per run, you could let the device be
programmable via a serial port on a PC, and write software to do that.
Or use a PIC with an RF interface (www.microchip.com). So you would not
have to detach and retach the device for programmability. You would
program via RF from a central machine.

One of 1000's of people who would be able to give you a more
exact/reliable estimate: http://www.zebryk.com/
[Disclaimer: I neither know, nor endorse this person.]

-Le Chaud Lapin-
 
J

Joel Kolstad

Jan 1, 1970
0
Peter Olcott said:
I need a hardware device that can feed keystrokes to the keyboard port, and
feed mouse actions to the mouse port using software to provide this data.

Why does it have to be hardware? Since you already specified that you're
running on Intel platforms, writing drivers for Windows will probably cover
95% of all your potential users, and if you toss in Linux and the Mac OS,
it's probably >99%...
 
D

Don Cleveland

Jan 1, 1970
0
Peter Olcott said:
I need to know how difficult it would be to design and build a device that
is able to force hardware keystrokes and mouse operations into an Intel
based motherboard. I was thinking that it could have one end plugged in to
a serial port or USB, and the other end would be a Y connection to the
keyboard and mouse ports. How difficult and costly would building such a
device be?

Peter,

You might want to look at

http://www.hagstromelectronics.com/


Don Cleveland
 
J

Joel Kolstad

Jan 1, 1970
0
Eeyore said:
What damn hardware ? You're very poor at explaining yourself.

One thing the OP may not realize is that, on all modern Windows OSes, you can
generally plug in as many keyboards and mice as you like and Windows takes
care of "merging" their input streams.

So I have a suspicion the OP really has no need whatsoever to build a
"wedge"-type piece of hardware, but rather just a device that emulates a
keyboard or mouse and can be (in some unspecified programmatic manner!) told
what to "type" or where to "move and click".

In my mind this would still make a lot more sense to do in software. If his
fancy technology can recognize what pixel needs to be clicked on anyway, a
Windows driver can trivially move the mouse pointer to that spot and "click
it."

---Joel
 
J

James Waldby

Jan 1, 1970
0
Jan said:
..."Peter Olcott"... wrote ...
"Jan Panteltje" ... wrote ...
Are you saying that doing a screen capture is hard, or recognizing the
graphical objects from the screen capture? (I got the last part solved).

I am saying tha tto capture the screen in Linux, you must have access to the
display buffer.
If it runs Xwindows, in an application a buffer is created [snip re X and buffers]
I would not know how to do it in a way that worked for _all_ systems known
on Linux.

This is assuming your application reads what is on the screen, what you
told me.

On any Unix system where the ImageMagick package is installed, one can
use "import" to get an image from any part of the X display. See eg
http://linux.about.com/library/cmd/blcmdl1_import.htm or man import.
It is straightforward to exec import from a C program and then read
the image file, which can be in any of the one-hundred graphics formats
that ImageMagick supports.
-jiw
 
J

Jan Panteltje

Jan 1, 1970
0
Jan said:
..."Peter Olcott"... wrote ...
"Jan Panteltje" ... wrote ...
Hi, I forgot to tell you in comp.os.linux.development.apps that the part
where you read the screen is also problematic, as X has its own graphics
drivers, etc.
Are you saying that doing a screen capture is hard, or recognizing the
graphical objects from the screen capture? (I got the last part solved).

I am saying tha tto capture the screen in Linux, you must have access to the
display buffer.
If it runs Xwindows, in an application a buffer is created [snip re X and buffers]
I would not know how to do it in a way that worked for _all_ systems known
on Linux.

This is assuming your application reads what is on the screen, what you
told me.

On any Unix system where the ImageMagick package is installed, one can
use "import" to get an image from any part of the X display. See eg
http://linux.about.com/library/cmd/blcmdl1_import.htm or man import.
It is straightforward to exec import from a C program and then read
the image file, which can be in any of the one-hundred graphics formats
that ImageMagick supports.

This is correct, and you can get window info to simply with xwininfo.
However the OP wants it for _all_ Linux GUI apps.
So there is svgalib, others, and I can also in an X program draw a menu
without using or informing the X server (of a subwindow), using fonts
unknown to the X server etc.
I have done this.
And you can have the situation where a person was in X, and then did
ctrl alt F1 to show a normal console, so simply testing if X is present
does no guarantee that is what is on the screen, etc.
Other languages, character sets, libraries, video fullscreen, we have it all.

What the OP told me he has, needs to read and evaluate what is on the screen.
'Impossible' may not exists, but it will be _very_ difficult to cover all
cases in Linux, it is not a monolitic integrated GUI like MS Widows.
 
Q

qrk

Jan 1, 1970
0
I need to know how difficult it would be to design and build a device that is
able to force hardware keystrokes and mouse operations into an Intel based
motherboard. I was thinking that it could have one end plugged in to a serial
port or USB, and the other end would be a Y connection to the keyboard and mouse
ports. How difficult and costly would building such a device be?

Is this a homework question? I haven't seen these devices for years!
The old macro keyboards that inserted between keyboard and computer
were probably under $100. Modern solution is use something like Macro
Express <http://www.macros.com/> which is much more convenient to use.
 
P

petrus bitbyter

Jan 1, 1970
0
Peter Olcott said:
These would be mass produced. It looks like you have the right basic
concept. How many hours would it take to build a prototype?

Hard to say because of the prerequisite constrains are unknown to me. I'd
say 80 to 100 hours. Half of that time would be used for the real work. So
making a ready to use proto (including enclosure, connectors, power supply
and so on). The other half for gathering information, like details of the
protocols, availabily of components and obtaining them and for making
decisions. After all, when you want to go to mass production, component
price and availbility are important issues.

For instance, when I need to be fast, I'd go for an ATmega162. It's like
killing a mosquito using a canon but it's the smallest one with two UARTS I
know for the moment and it's not cheap. And there you go. Looking for a
cheaper solution requires time so you're slowed down. I should not be
astonished if even this very posting starts a discussion about more simple,
cheaper solutions. Which is, after all, the main value of the group.

petrus biybyter
 
E

Eeyore

Jan 1, 1970
0
Joel said:
One thing the OP may not realize is that, on all modern Windows OSes, you can
generally plug in as many keyboards and mice as you like and Windows takes
care of "merging" their input streams.

So I have a suspicion the OP really has no need whatsoever to build a
"wedge"-type piece of hardware, but rather just a device that emulates a
keyboard or mouse and can be (in some unspecified programmatic manner!) told
what to "type" or where to "move and click".

In my mind this would still make a lot more sense to do in software. If his
fancy technology can recognize what pixel needs to be clicked on anyway, a
Windows driver can trivially move the mouse pointer to that spot and "click
it."

I would have thought so.

Graham
 
F

feebo

Jan 1, 1970
0
I need to know how difficult it would be to design and build a device that is
able to force hardware keystrokes and mouse operations into an Intel based
motherboard. I was thinking that it could have one end plugged in to a serial
port or USB, and the other end would be a Y connection to the keyboard and mouse
ports. How difficult and costly would building such a device be?


really, REALLY easy... basic one way serial code - provide the correct
key codes (these are the mappings for each key and not anything like
ascii values - also may be upto 3 bytes each) and bob's yer kipper -
google the PS2 interface adn that should give you everything you need.
 
F

feebo

Jan 1, 1970
0
I need to do the MOUSE and the KEYBOARD. I have patented technology that can
intelligently recognize anything on any GUI screen. There are some applications
that get their keyboard input directly from the hardware. There might be some
applications that do the same thing for the mouse. I want a single solution that
always works on the Intel platform that can be used for automated testing on any
OS.

!

you have a technology this advanced and yet you are having trouble
with the PS2 interface!?
 
A

Andrew Gabriel

Jan 1, 1970
0
So the Mouse PS/2 port would be easy, too? The device could use a regular
serial port instead of a USB, would this make it easier?

If this is something for future PC's, bare in mind PS2 and serial
ports will cease to be included on new systems. This is because
Microsoft has told PC manufacturers to remove all the leagacy
devices on the LPC bus (internal ISA bus), which are mainly the
built-in PS2 mouse/keyboard, parallel printer port, joystick port,
serial ports, floppy disk.
 
Top