Spiked3.com

Purpose

I recently did a project that allows you to drive a LEGO Robot with a webcam on it, in my case I used an Android phone camera. You drive the Robot remotely using an Android tablet, using tilt, and watching its webcam. I had done a similar project several years back, and wanted to update it, using newer technology. In the previous version I had written my own ‘webcam’ software and implemented it as a http web server. There are now good stand alone webcam packages available for Android phones (such as IP WebCam), and probably others as well, and I wanted to use them as opposed to my own somewhat hacked code.

I was able to use code from the LEGO MINDdroid application for the low level bluetooth communications to the LEGO NXT. This saved a ton of time, as they make the source available and have already done a lot of the hard work. As an experienced developer, I knew what I wanted to add, and this is where I struggled a bit, finding good working examples of what I wanted to do. I say experienced, but I was not experienced in Android development. At one point I wanted to draw my own control to display the joystick (tilt) position. The only thing I found was a full screen application with a particle system that displayed many three dimensional balls. I needed 5 lines of code, it had 2000. So while it may have the needed information, it was overkill as an example.
With that in mind, I present the results of my project, both as Android source code with some explanations, and the LEGO Digital Designer file to make your own robot from. Thanks again to LEGO for the NXT platform in general and making the MINDdroid source available. Thanks to Douglas Taylor, rChordata for his DiamondPoint algorithm (used and distributed with permission), the best differential drive algorithm I have ever seen.

The files

NXTTeleop

I’m a newb at packaging Android stuff up. So, if I messed something up, let me know. Both the Source and the LDD file are in this zip.

Things that did have good examples

These are items that I did find help on the web for, but are provided here as a quick jump start.

PreferenceManager

The preferences manager provides an easy way to store data between application start ups. Eventually you probably will provide a dialog to edit that data, but it is a good idea to have in mind what needs to be in it from the start. My app does not have a dialog to edit yet (not sure it will), but it’s mentioned as its something I thought about. It is easy to use

SharedPreferences preferences;
SharedPreferences.Editor preferenceEditor;

in your onCreate override add

preferences = PreferenceManager.getDefaultSharedPreferences(this);
preferenceEditor = preferences.edit();

sensitivityConstant = preferences.getFloat("sensitivityConstant", 4.5f);
throttleMS = preferences.getInt("throttleMS", 100);
ip_to_use = preferences.getString("ip_to_use", "192.168.1.113:8080");

later to change a value use;

preferenceEditor.putString("ip_to_use", ip_to_use);
preferenceEditor.commit();

What the hell is a ‘Toast’? Oh, a messageBox – let’s confuse experienced people and call it something else

A toast is just another name for a message box. There is not a lot more to say about it, other than I looked for messageBox (by that name) for hours and could not find it in Android. Well here it was.

Handling touch on a button (equivalent to onPressed as opposed to onClick)

At one point I needed to move the camera tilt. I initially thought, easy, use onClick, but that really was not appropriate for a button that starts a motion when pressed and stop when not pressed.

The trick is to use touch; Add a button in your layout (in my case with an id of tilt_up) then
after the content is set in your onCreate override;

((Button) findViewById(R.id.tilt_up)).setOnTouchListener(new View.OnTouchListener() {
            @Override
            public boolean onTouch(View arg0, MotionEvent arg1) {
                int eventType = arg1.getAction();
                if (eventType == MotionEvent.ACTION_DOWN) {
                    if (!actionMotorStarted) {
                        actionMotorStarted = true;
                        sendBTCmessage(BTCommunicator.NO_DELAY, motorAction, 50 * directionAction, 0);
                    }
                    return true;
                } else if (eventType == MotionEvent.ACTION_CANCEL || eventType == MotionEvent.ACTION_UP) {
                    if (actionMotorStarted) {
                        actionMotorStarted = false;
                        sendBTCmessage(BTCommunicator.NO_DELAY, motorAction, 0, 0);
                    }
                    return true;
                }
                return false;
            }
        });

Adding a menu

fairly straightforward, but again, short to the point examples are lacking; add a menu xml (my_menu.xml in my case) file to your project;

<menu xmlns:android="http://schemas.android.com/apk/res/android">
    <item android:id="@+id/set_cam_ip" android:title="Camera IP"/>
    <item android:id="@+id/set_nxt_id" android:title="NXT"/>
</menu>

then add these to your main activity class;

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        getMenuInflater().inflate(R.menu.my_menu, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        switch (item.getItemId()) {

            case R.id.set_cam_ip:
                selectIP();
                return true;

            case R.id.set_nxt_id:
                Intent requestNxtConnectIntent = new Intent(this, DeviceListActivity.class);
                startActivityForResult(requestNxtConnectIntent, REQUEST_CONNECT_DEVICE);
                return true;

            default:
                return super.onOptionsItemSelected(item);
        }
    }

Things I could not find good examples of, but finally got working, so presented here as examples

Custom control – draw a simple circle – I can take it from here.

I fought this one for more than a day. I just want a custom control to paint on and ALL the examples I found were overkill.
Add a class (that extends view);

package com.spiked3.rb;

import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.util.AttributeSet;
import android.view.View;

public class JoystickView extends View {

    Paint p = new Paint();
    public float x, y;
    int left, right;
    int myWidth, myHeight;

    final float xNormal = 10f, yNormal = 10f;
    final int A = -10, B = 10;

    public JoystickView(Context context, AttributeSet attrs) {
        super(context, attrs);
        p.setAntiAlias(true);
        p.setStyle(Paint.Style.FILL);
    }

    public void show(float x, float y, int left, int right) {
        this.x = -x;
        this.y = y;
        this.left = left;
        this.right = right;
        invalidate();
    }

    float normalizeFloat(float valueToNormalize, float inLow, float inHigh, float outLow, float outHigh) {
        float scaleRange = outHigh - outLow, valueRange = inHigh - inLow;
        return outLow + (scaleRange * (valueToNormalize - inLow) / valueRange);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        // motor power
        p.setColor(Color.GREEN);
        final int barWidth = 20;
        int a = (myWidth / 3);
        int b = a * 2;
        canvas.drawRect(a - (barWidth / 2), myHeight / 2, a + (barWidth / 2), (myHeight / 2) - left, p);
        canvas.drawRect(b - (barWidth / 2), myHeight / 2, b + (barWidth / 2), (myHeight / 2) - right, p);

        // joystick position
        p.setColor(Color.RED);
        int plotX = (int) normalizeFloat(x, -100, 100, 0, myWidth);
        int plotY = (int) normalizeFloat(y, -100, 100, 0, myHeight);
        canvas.drawCircle(plotX, plotY, 10, p);
    }

    @Override
    protected void onSizeChanged(int x1, int y1, int x2, int y2) {
        myWidth = x1;
        myHeight = y1;
    }
}

then add it to your main layout xml;

<com.spiked3.rb.JoystickView
            android:id="@+id/joystickSurfaceView"
            android:background="#FF404040"
            android:layout_marginRight="20dp"
            android:layout_width="352dp" android:layout_height="288dp"/>

Even my example has added ‘stuff’, I hope it doesn’t get in the way.

Webview of a webcam in your application

This one also took me a couple of days to figure out. Adding a webView is simple. The problem was I could not get it to display a motion web page from the web cam. The web cam ‘movie’ displayed properly in all standalone web browsers I tried, but not the web view within my application. Since this was critical to my application, it was a show stopper. Eventually while looking for something else I ran across a line of code that enabled JavaScript. Doh. It is not enabled by default, and there really isn’t any error message I have seen to hint at that. But these 2 lines of code did it;

final WebView webView = (WebView) findViewById(R.id.webView1);
webView.getSettings().setJavaScriptEnabled(true); // important!

Displaying a dialog for text input

This is another one I spent days on. I found examples, but they didn’t work. Many examples were poorly written. Even posts on StackOverflow offered almost identical solutions, but it turns out they were missing a critical piece. There is a built in function AlertDialog.Builder but the examples (and the function itself) all assume choices, lists, radio buttons. It turns out you CAN add your own layout to it.

create a new layout xml file (ip_address in my case);



<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
              android:layout_width="match_parent"
              android:layout_height="match_parent"
              android:orientation="vertical">

    <EditText
            android:inputType="text"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:id="@+id/address_edit_text"/>

</LinearLayout>

the use AlertDialog.Builder in your code, after inflating it (on a menu click in my case);

LayoutInflater factory = LayoutInflater.from(this);
        final View textEntryView = factory.inflate(R.layout.ip_address, null);
        final EditText ipAddress = (EditText) textEntryView.findViewById(R.id.address_edit_text);
        ipAddress.setText(ip_to_use, TextView.BufferType.EDITABLE);
        final WebView webView = (WebView) findViewById(R.id.webView1);

        new AlertDialog.Builder(this)
                .setIconAttribute(android.R.attr.alertDialogIcon)
                .setTitle("IP Address for WebCam")
                .setView(textEntryView)
                .setPositiveButton("Ok", new DialogInterface.OnClickListener() {
                    public void onClick(DialogInterface dialog, int whichButton) {
                        ip_to_use = ipAddress.getText().toString();
                        preferenceEditor.putString("ip_to_use", ip_to_use);
                        preferenceEditor.commit();
                        // java script full screen page from ipWebcam app
                        webView.getSettings().setJavaScriptEnabled(true);   // important!
                        webView.loadUrl(String.format("http://%s/jsfs.html", ip_to_use));
                    }
                })
                .create().show();

The part that was missing in examples I found was the critical need to use the inflated view in the call to findById. I thought I had tried that once before and it did not work, but I probably had something else messed up. I also learned that some cases Android expects all lower case letters in an ID name, but doesn’t really tell you when. I can’t tell you when, but if something looks right to me and doesn’t work, I’ll change the name to all lower case and most of the time it fixes the problem. Nice huh?

Robotic/Generic

Joystick to differential drive algorithm; DiamondToolbox

As an added bonus in my code you will find some support classes for Doug Taylor’s DiamondPoint differential drive equation. If you are doing anything in robotics that requires a joystick to control a differential drive, this is it. It outperforms all other algorithms I have seen. The others I tried could do forward and reverse OK, but came up short for turning. Doug’s algorithm will do true full power available turns. I tried 3 different algorithms out (they are all still in my code if you want to look) and I could see the results on the joystick custom control I wrote. You can see how the power is more ‘proper’ with the DiamondPoint Algorithm. Doug has graciously permitted me to redistribute my conversion to Java of his algorithm. The original discussion is here;
http://social.msdn.microsoft.com/Forums/en-US/roboticscommunity/thread/1fbec2bf-680b-4d4e-8b84-8f6502132727

a generic normalize function

It showed up in code a little earlier, but I needed a generic use all the time function to normalize values. I found some ‘psuedo’ code on the web for such, and many real code examples to normalize all values in a collection, but not something single value oriented. So I include it here, just as another time saving ‘just copy his’ function (see normalizeFloat in the JoystickView class above).