@Dana Batali has joined the channel
programming-git
track the state of the build












:slightlysmilingface: well — it is open sourced. do we have a license setup for our codebase?





more or less "anyone copy and use the code as long as the code says it came from us"




dana_batali: interesting error... https://travis-ci.org/Spartronics4915/2017-STEAMworks/builds/198096932

thanks for the pointer... i'll fix the weird characters in the comment... I'm not sure why builds didn't fail locally.











nice to see some traffic here! In prep for potentially two new subdirs, the challenge is to make sure travis is properly configured for those two new projects...

I think that configuration should be pretty much the same (I have a bit of experience with Travis, so it shouldn't be a problem). The only major consideration is that there could be some changes to the Powerup build process.


adding webhooks to our active builds: https://get.slack.help/hc/en-us/articles/232289568-Use-GitHub-with-Slack

here's a script that could be placed in a "cron job" - to trigger behavior, like auto-track/build:
#!/bin/sh
UPSTREAM=${1:-'@{u}'}
LOCAL=$(git rev-parse @)
REMOTE=$(git rev-parse "$UPSTREAM")
BASE=$(git merge-base @ "$UPSTREAM")
if [ $LOCAL = $REMOTE ]; then
echo "Up-to-date"
elif [ $LOCAL = $BASE ]; then
echo "Need to pull"
elif [ $REMOTE = $BASE ]; then
echo "Need to push"
else
echo "Diverged"
fi
NB: a 'webhook' would be a notification-based solution, but requires an online webserver to receive a post (that we'd need to configure).
Why? since we now have a private repo, we can't use the free version of travis. I understand that @Declan Freeman-Gleason has a travis account that can build from it's own private repos (do I have this right?). One solution, then, is to create a (private) mirror repo on declan's account that is continuously updated whenever the spartronics repo is changed. (cron job does pull from spartronics, then push to declan, which triggers a build, which sends build results to this slackchan).

Travis actually has built in cron jobs for builds... I can get that working now.




So what's the skinny on classpath? At one point Declan put it into .gitignore. Just did a pull and my local classpath is modified. I'm going to ignore it for now.

The skinny is that eclipse modifies this file "on a whim" and, unless I'm missing something, it's likely that the mods came from your eclipse, not one of these changes. At first we hoped that .gitignoring it was a good idea. We found this made it difficult to start new eclipse. But the values that are checked in are good and helpful in starting a new eclipse project. What we want is to prevent future un-intended mods to this file. I regularly employ git checkout -- .classpath in my activities, fwiw.

@Dana Batali I see that you're testing streaming webcam video from the Pi... I've been interested in using webRTC to stream video for our purposes, and I recently found a nice little tutorial/program targeted towards the Raspberry Pi that seems like what I'm looking for. I would test this myself, but only have a Pi 2 without a picam. It seems fairly straightforward (famous last words), so I'm going to post the link if you're interested in trying it out/have any feedback on it: https://www.linux-projects.org/uv4l/tutorials/custom-webapp-with-face-detection/ (You don't need to do the face detection part, of course).

did you look at the one I just checked in... Works fine, so I'm not sure we need to got to webRTC, since browsers don't understand that protocol without plugins... (correct me if I'm getting that wrong).

after a more thorough read of this, i believe its unique offering is a way to get your hands on the pixels before the browser does. I think our primary goal is to minimize latency, and the two primary sources of latency are video encoding on the raspi side and networking overhead. This doesn't seem to address either of those. It is really cool from a client-processing-images point of view!

I would argue that the unique offering of webRTC is that it provides an open and standard way to do real time communication, which is not necessarily what we need. We're optimizing for time of implementation and latency. On the other hand, there are a number of ways this could work better than an mjpeg stream for us (better support by browsers, easier to interact with through Javascript, compensation for noisy and lossy connections)... But none are really mission critical. At the end of the day, we should choose the first thing that works well enough (probably mjpeg). I just wanted to get this out there as a possibility :slightlysmilingface:




code as checked in isn't fully functional... I'm working on the kinks on a pi right now.

but next time you have access to the vision pi, please make sure it's up to 2018.0.1


@Darwin Clark: here's an initial sketch/test of robot code intended to point to the current vision target. Obviously we're not sending target data, and we will need to work out some kinks in both the protocol as well as the means to enter and exit the test tracking mode...
https://github.com/dbadb/2018-POWERUP/commit/226e55bc7080950b610547996c1cf7e4efd43af6

@Darwin Clark: heads up, minimal subset of code migrated to solution subdir. Also: i'd like to do a review of the current code and algo with you. Need to disable print statements, ensure that it works without drawing results to screen, etc.


perhaps that's as simple as setting ax and/or ay to -1000 (which is an invalid angle).

also: it might be worth sending this state to SmartDashboard as a vision subsystem message: `/SmartDashboard/Vision/State == "TargetAcquired"`

@Darwin Clark - here's the magic incantantion for streaming the results of an algo. Right now, I've wired in trivial hsvAlgo for proof of concept. Currently the defaultAlgo shows black in my home environment. I'll leave it to you to figure that out.


@Darwin Clark: one observation about the current defaultAlgo: I see that you reject any box whose area is < 3000 (pixels)... Given that our acquired image is 320x240 and a 3000 pixel area would be a square 55pixels on a side, it's possible you can't see far because you can't see small. thoughts?

Definitely valid, that was in place for the lab stage of development, about a month ago. Now that we have reached a state of solid single-target acquisition, that number can be tuned down a lot.


@Declan Freeman-Gleason - it appears that our travis auto builds are no longer available... Just in time :wink:



@Darwin Clark: action items from today afaik:
1. need a fix to raspi code to ensure that after the robot connects (and sets the vision target to 0), that raspi causes the cube-not-found value to be sent.
2. need to figure out why it wasn't centering on the cube, seemed offset to one side.
3. get your changes to Robot code cleaned up and checked in.

For the first point: I added a temporary solution of setting the default target values to 100, but I want to look more in depth about why the target was not getting sent over. I'm convinced that there is a logic problem with the system that only pushes a new DX over the pipe, so I need to find a more concrete solution for that.

@Darwin Clark Another action item: make the LEDs only turn on when vision is in use. Some judges are sensitive about having very bright lights on all the time.

note that we haven't spent any time tuning the position-control values and that these are what turnToHeading depends upon...

(i noticed that we have a non-trivial settling time on the rotation, but it may be good enough )





The high level controller was still causing trouble. `P` cannot get to the setpoint without oscillations. My first attempt to deal with this was adding `ffv`. This didn't seem to generalize well, but I suspect that we might have better luck with this if we start with the `ffv` first. My second attempt was to get to the setpoint with `I` and `P`, and later `ffv` this also had trouble generalizing (probably because of the `ffv`, just `I` and `P` would work, but not get to the setpoint fast enough, and `I` would eventually wind up too much. My last attempt involved increasing `P` to get to the setpoint (but it oscillated), and then increasing `I` to level out the oscillation and some kV (which I think is actually like a `D`.) This worked ok, especially once I added an IZone analogue. This worked ok, but it still wasn't as smooth as I would like. I think that starting with a high `ffv` (like at 1) might be a better way to go in the future. I'm unsure what this will do on the first robot. When I used the values we had for the first robot on the second one it behaved in a similar way, but seemed to oscillate more.

@Dana Batali By changing the path that was causing problems, we were able to get the PID to play nice. We tested every path except driving to the far switch, and (on the practice field) they all worked! Hopefully this will carry into our matches on the real field. Ronan and I are going to try to get a two-cube autonomous working, or at least make it so the robot drives to a cube for the drivers to pick up.

Great news! Keep an eye on battery for negative correlation... Keep retrying switch auto between two-cube





We were using the legacy GitHub integration, so I updated to the newer one which is much nicer to use (no adding webhooks to repos, etc.)




































































































I accidentally pushed my changes to master, which merged the PR. I force pushed so I could do another PR. Whoops.

@Declan Freeman-Gleason Would there be a robotstatemap for vision, or is robotstatemap exclusively for odometry systems? (I'm not inplying vision ISN'T odometry :) )

Robot state map really just tracks the velocity and position of the robot over time.

So I think there would be something else to track vision targets. You should look at the 2017 254 code to see how they did it.



most of this is @Declan Freeman-Gleason recent refactoring. I just did a code review and integrated.






























































































@Darwin Clark I see you have framerate set to 60 in this commit… Have you tried increasing to 90fps?

@Declan Freeman-Gleason Played around with that a little bit. Rasising FPS seems to have no effect (in raspi.py). Probabbly limited by opencv functions (but CPU spinning at only 50% ???)

~
What about ~commenting out OpenCV processing?~ setting algo to empty? Throughput should jump to 90Hz if it’s the processing speed holding us back, right?

What’s our methodology for measuring update rate on the output of the pipeline?




@Declan Freeman-Gleason 90hz is correct when not running any opencv methods. In regard to update rate, It's not unreasonable to add a fps / copmputation time feature, but we don't posses a scientific way to measure that at this time. The generals premise is that we don't need anything more than 30 fps in deployment, as it would just be unnessiary and could actually be damaging if data sent over the pipe has a spike.

@Darwin Clark Just to be clear, I'm not referring to the update rate of the vision preview stream, but the update rate of `processFrame`. It seems to me that the throughput of that should be as close to 90hz as we can get. With that in mind, is there any reason not to set the default for `self.args.fps` to 90?

Also, have you been able to compare `--threads=4` to the default single-threaded mode?

@Declan Freeman-Gleason I may be unsure of what you're asking... I agree with your statement about processFrame, and beyond necessity, there's no reason to not set it to 90...

I was just wondering if you've noticed any difference between mutli-threaded and single-threaded mode... I suppose that it would be hard to tell because we're not measuring throughput. It seems like it might be good to set up some FPS measurement capabilities, just so we know if we need to improve performance or not.





















































































































@Justice James @Noah Solomon There’s a small issue with this code that was just merged. I see that the timer reset was added to the appropriate place on line 156 of Climber.java as I asked, but the resets were not removed from the other spots in the system state switch statement in `onLoop`. These should be removed because they are redundant.



































































































































































@Declan Freeman-Gleason: if you have the time, please review. Goal is just to spend less time in each loop on telemetry.


Looks good. We should test both with round-robin and no `outputTelemetry` at all to see if either make a difference.

As another data point, I saw some people say that excessive NetworkTables use was causing overruns. A WPILib maintainer said it can definitely happen “if NT’s updateValues() call stalls”. He linked to this as what runs under the watchdog: https://github.com/wpilibsuite/allwpilib/blob/master/wpilibj/src/main/java/edu/wpi/first/wpilibj/IterativeRobotBase.java#L197-L260
Curiously, `SmartDashboard.updateValues()` is outside the watchdog.





























@Declan Freeman-Gleason @Jeffrey Tappen: check this out after syncing this change and starting dashboard.
http://localhost:5080/?demo=1#tab2


btw: hopefully our autos won't run into game elements the way this one does :wink:

Very cool! It slows down the other graphs on Firefox, but Chrome seems fine (on my laptop), although it is still a little jerky... I wanted to do this during break, but I didn't have enough time, so I'm happy it happened.
Maybe we should have someone extend it to display the position of vision targets from solvePNP.














@Darwin Clark can you verify that --debug works? For runPiCam, recall that all output is sent to a file. If you want to see the file 'spew', try this bash trick `tail -f yourfile`

@Dana Batali Alright, sounds good. Later today I'm going to check-in cleaner logging practices in runPiCam.py.



@Darwin Clark - i did the plumbing for this but leave it to you to do the central computations: I assumed in this checkin that you'd write the method `rectUtil.computeTargetOffsetAndHeightErrors` since you're the man of all rectangles!


















@Darwin Clark @Martin Vroom @Chris Mentzer: you may want to print this out and have it on hand in case of emergencies. Also: who is responsible for backup microsd cards?
https://github.com/Spartronics4915/Vision/blob/master/2019/CompetitionChecklist.md



















@Darwin Clark: since i don't have a LED rig at home, I've been struggling to get the algorithm to produce results in order to debug realPNP. I've devised a scheme to support multiple configurations and i'm now starting to get to the meat of things. I did feel the need to clean up your recently cleaned-up logging. I hope you'll agree the result is a little simpler and we only really need a single logging stream (to keep things efficient and easy to follow). I noted the magic "tail" command when you want to debug via spew.

basic idea of the config:
config.py is a file with some python "dicts". This is where tweaking of a parameters should belong and it allows us to support a variety of test configs without muddying up the code with comments and old values.


in theory, this shouldn't get in your way modulo any integration headaches it causes. That said: we shouldn't expend precious pre-first-match robot-time on a difficult integration. And we should verify that the previous behavior hasn't been botched


@Darwin Clark: a slightly disconcerting realization: when you run inRange via the maskAlgo, we convert to hsv first, so the range values are h,s and v. But elsewhere, like realPNP you don't appear to do the conversion, so the range values are probably being interpretted as b,g,r... Is this intended?


@Dana Batali The method rectutil.findRects() does the inRange calculation. My logic behind doing it the way I did was based around what each inRange is trying to accomplish. For maskAlgo, the objective is to only show the range values, not to find any rectangles, where as in other algos, it is fairly common to run an inRange to find all the rectangles in the frame, so I moved that repetitive section of code into rectUtil.

















If I can get one of the PI MicroSD cards that are configured correctly, I'd like to make an updated image. However, when I tried this previously, my computer claimed I needed to format the drive, which I was not about to do.






@Ronan Bennett: i roughed in a pressure gauge atop the field in the driver page of the dashboard. It's currently looking for a nettab entry: Robot/Pressure and currently assumes that valid values are 0-120, with good values starting around 95. The definition of good, here, is the color gradient which is defined in the layouts file. It will probably require a little tweaking to make it a "perfect" the driver experience. Let me know if you have questions or concerns. Martin has had some experience in Dashboard tech so will likely be able to assist if you require any changes.



@Darwin Clark, @Declan Freeman-Gleason - this new variant has the angle corrected, the value for theta has been "fixed", Theta = 0 means robot is perpendicular. The sense of positive vs negative is "negotiable", and can be fixed either in robot code or vision code (see line 294 of poseEstimation.py)


also: I would currently recommend that you use "couch" as the camera model assuming that we're not employing rotation to address camera-mounting issues. @Darwin Clark - i don't think you checked in your latest configs. Didn't they "unrotate" the camera for us?

@Dana Batali Yeah, they did. The only change was in the config file on the line which defines "rotation", set that to 0. Sounds good!

i think this means that in your config you're still using the camMat with the 1.3 multiplies in it. I don't think we want those, which is why setting cam to "couch" is recommended. I did add another called "theory" and the default is still your 1.3multiplied one











@Declan Freeman-Gleason @Ronan Bennett: heads up gradleRIO 4.1
also: a new method (and Tests!) for trivial on-the-fly path generation. I'll leave it to @Declan Freeman-Gleason to decide whether to employ it and where. The test shows usage.




This works and is probably a reasonable change, but it only really makes sense for straight paths... If you try to use an S-curve I'm pretty sure you won't get the path you're looking for.


And so if you are using 2 waypoints to request an s curve, we'll need to modify the logic slightly. Some examples: distance < 15 inches (couldn't do an s-curve in that distance probably), start & end heading within a tolerance (ie: straight), or just another parameter.





























@Declan Freeman-Gleason @Darwin Clark: here's a control (`lazycleanup`) over whether we keep multiple streams open simultaneously (`lazycleanup:true`). I've played with `lazycleanup:false` and it appears to work fine with a few caveats:
1. when you go from front to back, front will not be available for 5 seconds (due to the respawn behavior of the frc supervisor). I manually edited runCamera and set it to 2sec and was able to switch more quickly without troubles. When you do run into troubles, the video will be frozen and this message would appear in the browser debug console:`h264player.js:4428 WebSocket connection to '
2. i've checked in a change to the layouts file that makes this the default on the grounds that the dual camera feed we looked at at the end of yesterday's meeting, looked more chunky and more stuttery. The untested hypothesis here is that this will result in fewer dropped frames.
Comments, concerns, questions?




going from front to back, back appears in under 1 sec (unscientifically). The issue addressed by the "lazycleanup" was going back from back to front (immediately after front appears).












@Darwin Clark @Declan Freeman-Gleason: update on camera calibration:
1. my two cameras are quite different
2. the values i obtained for dbcam8 were "pretty accurate" with the camera matrix checked in to poseEstimation.py
3. the values i checked in for dbcam7 are less accurate (8in off at 100in distance)
4. i tried the dbcam8 values on dbcam7 and got better results, so it may be a good idea to try this matrix on the robot1 vision cam.
So... camera calibration is important and that we may need to have a different matrix for each camera. I'm not sure if we'll have enough time to do any of this with all the other stuff going on. Likely true that driver-cam validation is more important. And making sure all the electronics are stable more important still. If most of tonight's time in centered on robot-stabilization, it's possible we'll just be twiddling our fingers. Any sense of this? Let me know if you can.


@Darwin Clark: hey! it's my sense that if you flip vertically that you must also flip horizontally . Are you sure only the -vf is what we want?

@Dana Batali That occurred to me too...You probably also want flip, I chose not to wire it in untill I can confirm.


i just confirmed (and added a variable so I can run the same layout without the flip)

Could you bring the camera you know to be good today (dbcam8)? I think that while I might be working on robot stabilization you and Darwin could be working on getting good values. If we only have one camera that works then we might as well just use that (I don't think we have time right now to get a good matrix characterization scheme working before competition, but we might as use the one that works).

one more point on this thread:
if we only run a single drivercam (ie: lazycleanup: false), we can afford a bigger bitrate (and thus better image quality)














@Declan Freeman-Gleason: probably not worth your time during the match weekend. The new code is entirely related to paths and you can now both visualize and edit/design more paths (via texteditor of paths.js)

shouldn't cause any problems if it's synced but those are famous last words, so don't let the driverstation folks sync this without your approval. (cc-ing @Darwin Clark @Ronan Bennett)




@Dana Batali (cc @Declan Freeman-Gleason) Today Declan and I decided it would be a worthwhile use of time to get some image data from the robot #1 vision camera, and this is the result of running the calibration script. Few notes:
- The new profile is called GPCalib, and is recorded under calibration.md and poseEstimation.py
- The camera matrix for GPCalib seemed reasonable and didn't set off any red flags, however a few of the corelation coeficants had their signs flipped compared to the (current alleged) high-quality calibrations.
- I've commited all the images used for data as reference (to help with remote debugging this weekend) however in the future we'll remove them from the git repo.
When gathering reference data, our objective was to simply gather the largest variation of angles and distances we could in the pit, is this similar to what you did? If we were to re-gather data would you recommend a different strategy? What mentality produced dbcam8 calibration?

Great that you're finding time to gather the data!
Questions/comments:
1. the .gitignore currently prevents the checking in of images below the calibrate dir (to prevent accidents). If you do want to check in your images, you'll need to modify .gitignore or place them in a nearby directory.
2. the sign of the distortion coefficients are fine if negative, the fact that they are small numbers is a good sign. Did the "corrected images" look good? It probably goes without saying, but it's very important that your checkerboard be a rigid flat plane with no wrinkles, etc.
3. the idea of gathering the large variety of angles seems like the right idea. I think i was trying to get angles in which the checkerboard mostly filled the screen and included "scraping" views where the perspective effect (size variation with distance) was evident.
4. another sign of "goodness" is that if you collect different sets of images, how different does the matrix appear? Also: i see you didn't include the RMS number into calibrate.md (RMS is a goodness of fit measure, smaller is better). https://en.wikipedia.org/wiki/Rootmeansquare
But the big question: did it work with PNP to produce accurate measurements?






I don’t know why that would be. Maybe we should try listening on a more specific IP or the IPv4 analogue of :::: like 0.0.0.0

@Dana Batali We’re going to upload a script that prints out netstat -lntu, pstree, systemctl status ssh… Any command requests?


i just checked in changes to appRaspi.js with more diagnostics. And a force to listen only on the IPv4 interface.
The key command is: netstat -anp | grep 8080


we can also install our service on another port:
1. edit appRaspi.js
2. modify cam specs in layout file











sigh.. the node process appears to have become "detached". And the result is that uploaded.py is repeatedly respawning. Given that the process is running I don't understand why we can't see a camera feed?


here's what i expect to see:
`runCamera`
#!/bin/sh
echo "Waiting 2 seconds..."
sleep 2
exec ./startH264player.sh
`./startH264Player.sh`
#!/bin/sh
cd /home/pi/spartronics/Vision/h264player
node appRaspi.js


before the connection attempt we should see:
├─svscanboot─┬─readproctitle
│ └─svscan─┬─supervise───startH264player───node───6[{node}]
│ ├─supervise───netconsoleTee
│ └─supervise───configServer───8[{configServer}]
├─syslogd

after the connection is established it should look like this:
├─svscanboot─┬─readproctitle
│ └─svscan─┬─supervise───startH264player───node─┬─raspivid─┬─{HCEC Notify}
│ │ │ ├─{HDispmanx Notif}
│ │ │ ├─{HTV Notify}
│ │ │ ├─{VCHIQ completio}
│ │ │ ├─{vc.ril.camera}
│ │ │ ├─{vc.ril.videoen}
│ │ │ └─{vc.ril.videore}
│ │ └─6[{node}]
│ ├─supervise───netconsoleTee
│ └─supervise───configServer───8[{configServer}]

here's what `netstat -anp | grep 8080` looks like with an active websocket:
tcp 0 0 192.168.0.7:8080 0.0.0.0:* LISTEN 21917/node
tcp 0 0 192.168.0.7:8080 192.168.0.4:62137 ESTABLISHED 21917/node


`exec node appRaspi.js`
why can't we just be using the official/tested versions of these scripts?

That makes sense. I thought we had started from a vanilla image when we added the debug info. Anything else to do other than reverting that file to the original?

just tried this:
#!/bin/sh
echo "Waiting 2 seconds..."
sleep 2
#exec ./startH264player.sh
#export PYTHONUNBUFFERED=1
#exec ./startSleeper.py
cd /home/pi/spartronics/Vision/h264player
node appRaspi.js
and got this
├─svscanboot─┬─readproctitle
│ └─svscan─┬─supervise───runCamera───node─┬─raspivid─┬─{HCEC Notify}
│ │ │ ├─{HDispmanx Notif+
│ │ │ ├─{HTV Notify}
│ │ │ ├─{VCHIQ completio+
│ │ │ ├─{vc.ril.camera}
│ │ │ ├─{vc.ril.videoen+
│ │ │ └─{vc.ril.videore+
│ │ └─6[{node}]
│ ├─supervise───netconsoleTee
│ └─supervise───configServer───8[{configServer}]
Which means that my "exec" theory isn't "it".

btw: i'm currently running the version of appRaspi.js that i checked in an hour ago. It ensures that we're using tcp, not tcp6.

on the driverstation:
$ netstat -an | grep 8080
TCP 192.168.0.4:63546 192.168.0.7:8080 ESTABLISHED
on the raspi:
$ netstat -an | grep 8080
tcp 0 0 192.168.0.7:8080 0.0.0.0:* LISTEN
tcp 0 915 192.168.0.7:8080 192.168.0.4:63546 ESTABLISHED

note that each time you upload a script, frcvision rewrites the runCamera script. I tend to favor ssh + vi.

it occurs to me that given the fact that you saw a raspivid process in an earlier pstree, that things were actually working. Are you aware of the fact that the browser window receiving the pixels must be visible when the connection is made? Workaround is to "refresh from that window" (i wrote this up in the Vision/Checklist and went over this with Darwyn).

We need to deploy the version that only uses regular (non IPv6) TCP. We will make sure the window is refreshed.
By the way, the image I just uploaded was of `uploaded.py`, not the wpilib script that starts `uploaded.py`.

runCamera is key: if you upload a python script that is really a bash script, it's possible it's trying to invoke python on it!


good, and each time you press upload in the frcvision app it rewrites runCamera. so best not to do that









seems like you should log in and try to replicate
cd h264playerdir
node
require("express")











Ok, that worked. We're going to try out the IPv4 changes, and we also set it to use a port explicitly allowed by the FMS.







We got the rear camera working with your ipv4 changes and port 5805 instead of 8080!





























minor detail: i claim we should be writing docs in .md and not .html, curious what is afoot here.



@Declan Freeman-Gleason - can you clue me in on this new github-pages notification? Is there a public url to read our docs?

think i just answered my own question:
https://spartronics4915.github.io/2019-DeepSpace/
shows that a .md file can’t be meaningfully published as index.html.
I guess this is where @Binnur Alkazily’s expertise for generating website content via jekyll or some-such might be valuable

@Dana Batali will need to do bit more digging on how to do auto generation of index.html, but here is the auto html generation of poem.md
https://binnur.github.io/doc-test/poem
repo: https://github.com/binnur/doc-test
Process:
1. settings, enable github pages
2. in .md file include `---title: page-tag ---` block to auto generate html

I haven’t digged into custom domains, so it is hosted off my 1st github page setup














