This page lists out some of the projects that I worked on prior to full time work at my current company. This is a mix of personal projects (my own ideas - such as Liszten), projects done via school but fully creatively designed by myself (such as Modal Space), or via former positions (e.g. as former lead mobile developer at Keewi Inc).
Liszten (Swift)
A groundbreaking app for musicians, music lovers, music students, and experimenters.
This app transcribes a melody into music notation; the corresponding audio is saved for playback.
Features:
Starting out as a fascinating idea bouncing around my head in high school, this rather daunting idea has finally come to fruition in the form of a mobile app. After a semester spent researching the most feasible technologies to use, myself (project manager and lead) and the rest of my team of four developed a MVP for this concept, which won the Kickstarter grant from Mobile Developers of Berkeley in 2017.
We had very basic pitch detection available via AudioKit (very raw, and discrete information - you can call a method to grab the most prominent frequency detected through the mic at any given instance), which I developed an algorithm to process and transform into information about notes and their durations in seconds, which is then passed through a rhythm detection algorithm (which is currently based on a very rough heuristic) myself and team member Lisa worked on, to generate information about how many beats each note is held and process it into sheet music.
There is a lot further to go with this app, especially in the areas of rhythm detection and sensitivity to natural fluctuations in pitch, but I am extremely proud of what we have accomplished in its first iteration!
It is available on the app store: https://itunes.apple.com/us/app/liszten/id1228544001?ls=1&mt=8
This app transcribes a melody into music notation; the corresponding audio is saved for playback.
Features:
- take any monophonic melody and…
- record as audio
- view transcribed notes in real time
- transform to standard sheet music notation
- export sheet music as PDF
Starting out as a fascinating idea bouncing around my head in high school, this rather daunting idea has finally come to fruition in the form of a mobile app. After a semester spent researching the most feasible technologies to use, myself (project manager and lead) and the rest of my team of four developed a MVP for this concept, which won the Kickstarter grant from Mobile Developers of Berkeley in 2017.
We had very basic pitch detection available via AudioKit (very raw, and discrete information - you can call a method to grab the most prominent frequency detected through the mic at any given instance), which I developed an algorithm to process and transform into information about notes and their durations in seconds, which is then passed through a rhythm detection algorithm (which is currently based on a very rough heuristic) myself and team member Lisa worked on, to generate information about how many beats each note is held and process it into sheet music.
There is a lot further to go with this app, especially in the areas of rhythm detection and sensitivity to natural fluctuations in pitch, but I am extremely proud of what we have accomplished in its first iteration!
It is available on the app store: https://itunes.apple.com/us/app/liszten/id1228544001?ls=1&mt=8
Sustainable Energy (Objective-C, Python/Django)
This past summer, as the lead mobile developer of a small startup focused on sustainable energy efficiency, I did some major work on their mobile app. The images and details included here are used with the company's permission.
My impact on the iOS app included everything from updating the UI and resolving backend issues, to getting the app through the App Store for its initial release, to designing and creating both the front end and backend for new, key pages and groundbreaking features.
Some features have been released in version updates on the App Store, and some are in Beta.
Above I have shown a couple pages that showcase some of the main features that I have worked on and/or created from scratch.
Among the pages that I worked on are the Progress > Points and Stats pages. For these two pages, I created designs in Photoshop, scouted out the most flexible data visualization libraries, and coded up both of these pages in Objective-C, as well as the menu slider that connects them. Along the way, I made several modifications to the backend code for the Stats page to correct and clarify certain calculations/provide more accurate data.
Perhaps a yet more important aspect of the app that I worked on is an up and coming automation feature (details of which I am not disclosing until it is published, for the company's privacy). For this feature, I created the software diagram and the timeline’d plan, and after several days of research, began cranking through all my outlined phases of implementation. In the end, I managed to implement all the backend/API’s, the mobile front end code, and UI for the essential functionality of the feature. For the sake of the company’s privacy, I will not go into too much detail about this feature and have not included screenshots of the feature, but I will say that it is pretty cool, and involves region monitoring, the collection of data (with the user’s permission and for the user’s sake), and guarantees that you will only use energy when you need it.
The other pages of the app have all been touched by my work in some shape or form, from the implementation of the slide menu’s consumption indicator, to the necessary backend changes to calculate the data for the History graph, to UI work all throughout the app, and numerous other UI/backend changes.
My impact on the iOS app included everything from updating the UI and resolving backend issues, to getting the app through the App Store for its initial release, to designing and creating both the front end and backend for new, key pages and groundbreaking features.
Some features have been released in version updates on the App Store, and some are in Beta.
Above I have shown a couple pages that showcase some of the main features that I have worked on and/or created from scratch.
Among the pages that I worked on are the Progress > Points and Stats pages. For these two pages, I created designs in Photoshop, scouted out the most flexible data visualization libraries, and coded up both of these pages in Objective-C, as well as the menu slider that connects them. Along the way, I made several modifications to the backend code for the Stats page to correct and clarify certain calculations/provide more accurate data.
Perhaps a yet more important aspect of the app that I worked on is an up and coming automation feature (details of which I am not disclosing until it is published, for the company's privacy). For this feature, I created the software diagram and the timeline’d plan, and after several days of research, began cranking through all my outlined phases of implementation. In the end, I managed to implement all the backend/API’s, the mobile front end code, and UI for the essential functionality of the feature. For the sake of the company’s privacy, I will not go into too much detail about this feature and have not included screenshots of the feature, but I will say that it is pretty cool, and involves region monitoring, the collection of data (with the user’s permission and for the user’s sake), and guarantees that you will only use energy when you need it.
The other pages of the app have all been touched by my work in some shape or form, from the implementation of the slide menu’s consumption indicator, to the necessary backend changes to calculate the data for the History graph, to UI work all throughout the app, and numerous other UI/backend changes.
YumFeed (Swift, Parse)
YumFeed is an iOS app for food-lovers, released on the app store at < tinyurl.com/yumfeed >. I worked with three other team members of Mobile Developers of Berkeley (a community of passionate, talented people interested in mobile development) to take our app idea from designs to app store completion in only 24 days of hard work, learning, and inventive problem solving! We gave ourselves such a time constraint in order to have our app published and ready to download in time for the Berkeley App Fair.
The app allows food-lovers to bookmark homemade items or sweet local food as "To Try" and mark them as "Tried" when done. Each person's profile displays three tabs: Posts, To Try, and Tried, so the social element of discovering what your friends have tried helps motivate the app. Recipes are provided for homemade items and restaurant information for foods discovered locally.
One app store review noted that YumFeed, "combines the usefulness of Yelp with the aesthetic simplicity of Instagram" (reviewed by ginakkiim).
The app allows food-lovers to bookmark homemade items or sweet local food as "To Try" and mark them as "Tried" when done. Each person's profile displays three tabs: Posts, To Try, and Tried, so the social element of discovering what your friends have tried helps motivate the app. Recipes are provided for homemade items and restaurant information for foods discovered locally.
One app store review noted that YumFeed, "combines the usefulness of Yelp with the aesthetic simplicity of Instagram" (reviewed by ginakkiim).
Modal Travel (Max/MSP, odot)
Programmed in Max/MSP with cnmat odot, modal travel is a conceptual musical piece, a synthesizer, and an instrument that transcends various musical barriers, allowing it to interact with the performer as he or she performs.
More specifically, the camera follows the objects in view, and based on their color, size/proximity, number, and location on an x y plane, determine a number of things such as base pitch (the "tonic" of the mode, in Western terms), musical mode (e.g. major, minor, phrygian, pentatonic, selisir), clarity of sound, and amplitude, fluidly morphing a recorded or live melody from the user. Each key of the keyboard creates a different sound depending on all these factors, but like a traditional keyboard, higher pitches are to the right, and lower pitches are to the left. Control of the musical mode is more direct than the rest of the continuous features, with the instrument set to follow the largest, reddest, greenest, or bluest object, depending on the user's choice.
With this delightfully avant-garde digital instrument, musicians can create abstract music to their heart's delight!
In all seriousness, though, I myself have discovered some beautiful sounds just jamming away on this. The fluidity of modes and sounds means that the relationship between key and pitch is looser than on most conventional instruments, and in one melody one easily flows between different little worlds of sound without changing the keys one is pressing.
Above is a screenshot of the patch, with myself using it (most of the code is abstracted away so only the elements needed for performance are shown). In this shot, I have set the patch to follow the bluest object on screen, and so the musical mode follows the movement of my phone, the bluest object in the camera's vision.
Repo (with more description)
Insomnia (Javascript, CSS, Html)
My first go at learning a bit of web design, insomnia (or nap sweetly) is a website I created to help people nap properly. 'Cause people do mess it up all the time and end up more tired than rested after napping, timing naps so as to wake up in the middle of a sleep cycle, or nap for too long in general.
The basic idea is to allow people to choose a proper nap length (that will not lead to sleep inertia or other undesired effects) based on the kind of energy they need. A quick boost? Flexible 10-20 minute power nap. Catch up on sleep and get your brain thinking creatively again? 90 minute nap, aka the full sleep cycle. Alertness of an astronaut? The highly acclaimed 26-minute NASA nap, which wakes one up right before one enters deeper sleep, getting the longest short nap possible without suffering from grogginess. Scrolling over the desired effect shows the length of the nap and a bit of science behind it. The poor tired soul using the site can finally get a proper nap; one click on the kind of nap desired begins an alarm clock set to the exact nap length of the kind of nap chosen!
Definitely still learning, but this is my first bit of web stuff ever written, so I thought I'd include it here!
Repo
University of California, Berkeley
Here are some of the cooler project assignments I've worked on through the coursework at Cal:
Here are some of the cooler project assignments I've worked on through the coursework at Cal:
Face Morphing (CS 194-26)
See the website I built for this project here.
In this assignment I produce a "morph" animation of my face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.
A triangulation is then defined on the correspondence, and pixels in each triangle are warped to the desired midway position using an affine transformation that I compute as a change of basis matrix based on the triangle vertices.
Written from scratch in python3.
In this assignment I produce a "morph" animation of my face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.
A triangulation is then defined on the correspondence, and pixels in each triangle are warped to the desired midway position using an affine transformation that I compute as a change of basis matrix based on the triangle vertices.
Written from scratch in python3.
Frequencies and Gradients (CS 194-26)
See the website for my project here.
Part one of this project demonstrates image sharpening, hybrid images, creating Gaussian and Laplacian stacks of structured images, and using these stacks to spline images via multi-resolution blending across different frequency bands. The different levels of the Laplacian stacks are convolved with Gaussian filtered masks in order to achieve the blended image.
Part two of this project focuses on gradient domain processing, specifically Poisson blending, to combine parts of images with others smoothly.
Written from scratch in python3.
Part one of this project demonstrates image sharpening, hybrid images, creating Gaussian and Laplacian stacks of structured images, and using these stacks to spline images via multi-resolution blending across different frequency bands. The different levels of the Laplacian stacks are convolved with Gaussian filtered masks in order to achieve the blended image.
Part two of this project focuses on gradient domain processing, specifically Poisson blending, to combine parts of images with others smoothly.
Written from scratch in python3.
Image Warping and Mosaicing (CS 194-26)
View my project website here.
In this project I take two or more photographs and create an image mosaic by registering, projective warping, resampling, and compositing them. Along the way, I compute homographies, and use them to warp images. In the second part of this project, automatic feature detection is used to completely automate the process of mosaicing.
In this project I take two or more photographs and create an image mosaic by registering, projective warping, resampling, and compositing them. Along the way, I compute homographies, and use them to warp images. In the second part of this project, automatic feature detection is used to completely automate the process of mosaicing.
Early Projects
Here are some of the projects I worked on in when I was first introduced to coding, in late high school.
Here are some of the projects I worked on in when I was first introduced to coding, in late high school.
Minesweeper with Autoplay (Java)
After a little attempt at GUI on netbeans, this minesweeper game was coded entirely from scratch in java! One of my first java projects involving GUI. Operates just like any minesweeper would, with the usual timer, restarting smiley, sound effects, and difficulty level. I added some flair of my own; different visual settings and computer auto play mode (and yes, the computer can lose)! The computer doesn't "know" anything more than a user would know. So, using the same logic a regular player of minesweeper might use, the computer player "clicks" on the squares that are guaranteed to be bomb-less, and flags the ones guaranteed to be bombs, progressing through the game like any old user. The only difference is that the computer can click multiple squares at once. To make it more clear what steps are being made, the squares to be clicked by the computer are highlighted green before they are revealed. This project, one of my favorites from that first year I began to code, integrated a lot of ideas I had just learned at the time, from intense recursion (e.g. in the revealing of a section of squares from one click) to heavy use of graphics (java swing, java awt), to efficiency (no one wants to play a slow game) and more.
Repo
Game of Life (Java)
The photos don't really do the game justice in this case, as it is mesmerizing to watch the animation. This java GUI application, coded from scratch, again with java's graphics package, simulates Conway's game of life. The user draws anything they like, and begins the animation, and based on the rules of the game of life, cells live, prosper, reproduce, and die.
Though I have included an option for single-step stepping through the generations of life, I prefer watching the continuous ebb and flow of the cells in continuous mode. The slider adjusts the speed of the continuous animation. There is a custom color palette for the cells offered in addition to red, green and blue standard choices. If the color is changed in the middle of the game, then the old and new color coexist until all the cells of the old color have been touched somehow. If a cell is static, it does not change color. The user can continue drawing onto the grid new cells even if the game of life is currently many generations in and animating continuously.
Repo
Though I have included an option for single-step stepping through the generations of life, I prefer watching the continuous ebb and flow of the cells in continuous mode. The slider adjusts the speed of the continuous animation. There is a custom color palette for the cells offered in addition to red, green and blue standard choices. If the color is changed in the middle of the game, then the old and new color coexist until all the cells of the old color have been touched somehow. If a cell is static, it does not change color. The user can continue drawing onto the grid new cells even if the game of life is currently many generations in and animating continuously.
Repo