@Jennifer Marsman has joined the channel
@Jason Holmberg (Wild Me) has joined the channel
@Stefan Schneider has joined the channel
@Elijah Cole (Deactivated) has joined the channel
@Lily Xu - OLD ACCOUNT, DO NOT MESSAGE has joined the channel
@Christine Kaeser-Chen has joined the channel
This here is maybe some borderline hodge-podge: How do you all figure out and hear about conferences and workshops that might have good conservation technology related material? I have a list of a dozen or so old standbys (e,g. AGU, Living Planet Symposium) but are there any good comprehensive resources?
i second this. unlike tech conferences (which seem to have meta-listing pages as well as are plentiful, the conservation space seems difficult to find out about. maybe a mailing list or something?
example: this one took place in the city where i live and i only heard about it via a colleague who was going and happened to mention it to me!
http://urban-wildlife.org
Agreed! This is one of the main reasons I started this group 🙂
But to answer your question Dave, I mostly hear about them through the grapevine, via colleagues.
New book from the Natural Capital Project: “Green Growth That Works: Natural Capital Policy and Finance Mechanisms Around the World”
Goodread: https://www.goodreads.com/book/show/44451605-green-growth-that-works
Island Press, ships worldwide, use code GROWTH for a 20% discount: https://islandpress.org/books/green-growth-works
Facing the one pixel attack or others (anything is an ostrich 🙂 ) or the work of Marcus at CoRR 2018 (Deep learning: A critical appraisal), I was wondering why CNNs are still used even in computational simple approaches where alternative models (bayes methods) or linear models (e.g.: PCA) would perform great. To me, it seems, that the simplicity of both, the math of aNN and the usability, leads to applications where GPU based CNN methods are not necessary... (In Austria we say: shoot for sparrows with cannons 🙂 ). I would like to hear your opinion. Any comments?
@Wilfried Wöber for me one of the major advantages of CNNs is their ability to learn their own features, not something you easily get from other ML methods
@Laurens Hogeweg yeah that makes CNNs somehow sexy to use facing matrix-like struktures... On the other hand, graphical models or conditional exponential families are (theoretical) able to do similar stuff but are limited in terms of computational costs... To me, CNN application seems to have more practical advantages as statistical benefits. For example, Neal discussed that probabilistic MLP using infinite neurons becomes Gaussian processes.
In case someone of you is interested in pose estimation of the western honey bee, check out our new dataset:
https://github.com/apic-ai/apic-bee-pose-dataset
The apic.ai bee pose dataset contains annotation of 32 relevant keypoints on the western honey bees. Explicitly inferring the pose is beneficial for the identification of behavioral anomalies such as trembling after poisoning or infections.
This is fantastic! Congratulations.
Anyone with more tf.data experience have an idea? Trying to catch individual corrupted files in tfrecords batches https://stackoverflow.com/questions/61735361/how-to-catch-and-raise-tfrecord-error-in-tf-dataset-map-in-tensorflow-2-0
I use the following approach (this is taken from a file download script):
# Ignore the images that fail, we'll detect this by comparing all the images that succeeded with the full list of images.
dataset = dataset.apply(tf.data.experimental.ignore_errors())
dataset = dataset.as_numpy_iterator()
successful_downloads = []
for filename in tqdm(dataset, total=num_downloads, mininterval=3):
successful_downloads.append(filename.decode())
The idea is to ignore all errors and have your dataset return filenames. If a filename was returned then you can assume it was not corrupted
then I do a diff against all the filenames to find the corrupted ones
I’ve started to save the image files themselves using tensorflow in order to cut down on errors when I subsequently store those images into tfrecords for later experiments
def save_jpeg(image_buffer, filename):
image = tf.io.decode_image(image_buffer, channels=3, dtype=tf.dtypes.uint8, expand_animations=False)
image_shape = tf.shape(image)
jpeg_buffer = tf.io.encode_jpeg(
image,
format='rgb',
quality=95,
progressive=False,
optimize_size=False,
chroma_downsampling=True,
density_unit='in',
x_density=300,
y_density=300,
xmp_metadata='',
name=None
)
tf.io.write_file(filename, jpeg_buffer)
return filename
Hi everyone. This doesn't exactly fit into the remit of the channel but I hope you don't mind me sharing it none the less. I have some colleagues that are trying to collect a dataset of dog images so they can develop future AI tools for monitoring wild dog populations with the aim of tracking the spreading of diseases like rabies. Any retweets would be greatly appreciated to get some more eyeballs on it. Thanks!
https://twitter.com/fieldinghr/status/1304043176987176963
Anyone have good references for expert drawn species range maps? I'm writing a review, and I'm looking for literature on the pros/cons/biases of hand drawn range maps
Are you looking for published references only? (my wife) Janet created all the range maps for her recent book on Camera Trapping. What a pain! I'm sure she has some ideas on pros/cons/biases (sources of error, etc.) she would share. Referenced as "personal correspondence". There must be a better way!
Wow!! Yes, I'd love to talk to her! I'm writing a review paper and we're already referencing a few interviews
Hello. I am conducting a Conservation Technology Research Project with Babson College and would appreciate if any of you fine folks would take a brief survey: https://www.ellenmalloy.com/survey/
If you haven't already, you should definitely connect with the WILDLABS folks as they are doing a big 'state of conservation tech' project right now!
https://www.nytimes.com/2020/11/11/science/bears-facial-recognition.html
This is @Ed Miller's work! Awesome to see it go viral!!!
We were very happy to see our paper picked up by the NY Times! They wrote a nice article as well.
Anyone with experience moving from tensorflow to pytorch? I'm worried the time has come, 2 of my upstream repos have deprecated and jumped ship.
lol, you were the one person I thought wasn't going to say that.
keras-retinanet deprecated, moved to torchvision https://debuggercafe.com/object-detection-using-retinanet-with-pytorch-and-deep-learning/
@Benjamin Kellenberger and I were talking about this recently
i've spoken to the developer and I trust his opinion and he was just not able to get the same results tensorflow 2.0
i was planning a full reboot for deepforest in the spring
Context r-cnn results don't match in tf2 either
I've been banging my head against it for a while
i'm worried about losing tf.data and tfrecords
That sounds familiar, except that I’m having that problem the other way round (I may need to get familiarized with TF but don’t want to dive into their obfuscated logic).
For off-the-shelf detectors & Co. I’ve made good experience with Detectron2. It doesn’t include everything out of the box, but can be extended easily (I e.g. got an InceptionResNetV2 backbone implemented within minutes).
I'm still learning the ropes, but I made the switch and I found it easier to pick up. I personally find it much easier to work with and debug. I also find dataloader in pytorch much more sophisticated than tfrecords or tfdata. I think the main argument for tf has been the enterprise features that it supports, but I'm not really interested in that as of now, and (don't quote me) I think pytorch has addressed that by now too. Granted, I'm not yet doing as advanced things as I presume you guys are doing so take my opinion with a grain of salt.
thanks. day 4 with it and liking it. I haven't gotten to the multi-gpu dataloader questions. I am worried I will miss tfrecords, my logic being that since pytorch is more python, it feels like it can't be as fast as binary read from tfrecords, but we will see. We have pretty big data.
Nice! I can't comment on my own experience with it, but a researcher on a reddit exchange commented that his own benchmark tests showed pytorch dataloader to be as efficient as tensorflow tfrecords. But, I am interested to here how it goes for you as well. Good luck with everything!
Correct me if i'm wrong but I'm pretty sure you can write a custom pytorch dataloader class that uses tf.data internally
found this on a random japanese blog post which also comments on performance later on https://qiita.com/namahoge/items/71be06c8fe37e88909c9
Thanks. I guess my question is what strategies to use in pytorch to try to get similiar performance. Whether a dataloader will just get IO bottleneck because of the python GIL. Its all nice to be more readable because its python code, but i'm worried about scaling up training speed. No good to have multiple GPUs if you can't pipe data in fast enough.
@Ben Weinstein
I haven’t gotten to the multi-gpu dataloader questions. (…) I am worried I will miss tfrecords, my logic being that since pytorch is more python, it feels like it can’t be as fast as binary read from tfrecords, but we will see.In the past TF has been faster than PyTorch due to JIT C compilation. However, PyTorch now offers the same and speed differences are getting minuscule.
Unlike TF, execution by default happens in Python, apart from core functions (e.g., convolutions and the autograd backward pass), which gives full transparency, but as said with JIT performance can be optimized.
Regarding multi-GPU training: the fastest way of doing that is to use the DistributedDataParallel wrapper (essentially a one-liner).
Regarding data loaders: I don’t know what TF records specify. Perhaps they’re optimized for speed, which is great. The PyTorch approach instead allows you to load, prepare, manipulate and return data records in any form you want. The standard way of doing is is to write a Dataset wrapper class that (i.) loads your data (images, etc.), (ii.) applies Transforms (built-in or any custom one) and returns data. The DataLoader class then accepts a dataset class instance and takes care of multi-threaded data loading. There’s no GIL here; PyTorch extends the Multiprocessing module and allows proper multi-threading (flag num_workers in the DataLoader class).
There exists a library called PyTorch Lightning that helps with normal DL operations -- https://pytorch-lightning.readthedocs.io/en/stable/new-project.html
There's wrappers that allow you to use your tensorflow code in pytorch. Something like (https://github.com/BlackHC/tfpyth). It's definitely more for prototyping than serious development, but might be worth a shot.
Hey guys I noticed that there were no subreddits dedicated to computational sustainability. So, I decided to make one. You can find it on reddit.com/r/compsust. It's pretty barren right now, so it would be nice to make it more active. Also, because I am new to the field, if anyone would like to become a moderator, just let me know!
My everglades bird detector hit mAP = 0.8 on 15,000 nesting egrets and ibis! Post other recent wins here!
Does anybody have any interesting/must read papers related to the long-tail distribution problem with CV that they would recommend reading? I’m trying to read up more on it but I want to make sure I’m not missing anything important.
A few good keywords: data imbalance/imbalanced data, few-shot learning, low data regime
I really like this paper by Yin Cui: https://openaccess.thecvf.com/content_CVPR_2019/html/Cui_Class-Balanced_Loss_Based_on_Effective_Number_of_Samples_CVPR_2019_paper.html
Thanks for pointing out that paper @Sara Beery--looks very good. I’ve only skimmed it so far but it looks to me as if they are focused on single-label classification. Do you know if their method also works for multi-label classification, i.e., when you have more than one class in an image?
I think you could apply it in a multiclass scenario if you have localization, so each bounding box for example has one correct class. I haven't seen a direct multiclass version, but one may very well exist 🙂
Just learned about OpenCV’s new OAK board. I was wondering if anyone in this group with camera trap experience could comment on the efficacy of using these in the future.
I only got some experience with the kendryte k210 and was very impressed by its low power consumption and price.
30USD for display, camera and board.
I could not find any numbers but the OAK should be way more energy hungry (i might be wrong on this)
That makes sense. So you would say in your experience the most important factor should be power consumption?
I haven’t heard about the k210, but it looks very good.
I also found the estimated specs on the OpenCV boards. 2.5W for the OAK-1 and 4W for the OAK-D. So, you’re definitely right in that they are far more power hungry than the 0.3W k210.
Hi all, anybody have specific recommendations for an external hard drive for photos? Thank you!
I’ve always used Seagate external hard drives!
Also consider an SSD instead of an HDD -- slightly more expensive, but faster and less likely to destroy your data if you drop it
@Dan Morris can you put https://github.com/weecology/NeonTreeEvaluation (plus training data https://zenodo.org/record/4746605) on http://lila.science/otherdatasets
Hi there! I'm conducting a UX research project pertaining to onboarding and technical software use and would love input from anyone who has time for a quick 10-15 minute zoom/phone interview! Please feel free to DM me if you are interested in helping out 🙂
If you are developing ML on microcontrollers or planning any tineML or AIoT, please take a few minutes to respond to this survey, developed by Arm with support from our ecosystem partners. https://www.surveymonkey.co.uk/r/ArmML-TinyML
I've been building this list of conservation tech resources for folks who are interested in working in conservation tech or generally getting involved, and so far it's been a helpful thing to be able to share with folks. However, it's definitely not a comprehensive list, and I know many of you would have great stuff to add. So if you're interested, please feel free to suggest more organizations, educational resources, communities, open source projects, and anything else you think should be on here. And, feel free to share the list if you know of anyone who might find it to be helpful! https://docs.google.com/spreadsheets/d/16RVh2J1cgmiMjCgIBoM9uU3ADXQXtdJ2GuLf1_ToHLI/edit?usp=sharing
this is super cool, thanks so much Gracie! sharing this on other related networks (CompSust and MD4SG environment group)
ooo I have a spreadsheet of my own that I’ve been using for personal reference but I’d love to merge them! @Gracie Ermi could I email you the doc so you can just copy/paste over? Probably more efficient than me filling out the form a bunch of times. I can re-format it to match your column names
@Carly Batist absolutely! you can send it to grace.ermi@gmail.com
great initiative ! Could you please add a few lines about our organisation as well?
organization: Sensing Clues Foundation
vol/tech-skills required: OpenEars / SERVAL open source sound recognition project)
CAIMAN open source cam-trap algorithms for recognising species and counting individuals > see kaggle iWildcam 2021, where we became photo-finish second
Conservation-tech:
WITS turn-key open platform for nature conservation and wildlife protection:
- Cluey Data Collector & Tracking app
- Focus for Ops Rooms and google-like search across data sources
- datahub (semantic triple store, geospatial, and doc-store, highly secure)
- WildCAT for end-users (various apps) and for researchers (fully integrated Python, R, Jupyter Notebooks, Shiny, etc.)
- CAIMAN (image recognition, see above)
- SERVAL (sounds recognition, see above)
Curious to hear people's thoughts on how to avoid this sort of situation in AI for conservation?
https://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/
Test in realistic scenarios, don't make unvalidated claims, build systems that allow end users to continually re-evaluate (ie quality control) to catch issues/model drift early on?
In a sense, this is an old problem with research. Academic researchers have few career incentives to share work or validate existing results. There's no reward for pushing through the last mile that takes tech from "lab bench to bedside," says Mateen.
This has been our experience as engineers adapting academic techniques.
We read the claims, but the devil is not only in the training data but also in the implementation and domain.
And sometimes we don't even ask the right questions until we've applied the models several times to have the experience and context to understand.
I wonder how many of these "hundreds of models developed" pretty much do the same thing, badly? That's a huge waste of human and financial capital. It seems like this was unavoidable wrt covid... We faced (face?) a large and immediate existential threat, which meant a LOT of financial motivation to abate the risk. I think it's understandable, if not expected, that people turned to a little understood technology with a consistent track record for misleading claims and just threw money at the wall hoping that we'd come up with a magic bullet. I feel that better coordination of efforts in general is a huge part of the solution. Open data standards, an expectation and insistence for peer review for public and environmental health models, and stronger coalitions of academic researchers, agencies, regulators, and industry professionals who can combine their resources instead of operating all independently in silos, all with different capabilities distributed across the TRL curve.
"I feel that better coordination of efforts in general is a huge part of the solution." 💯💯
In conservation, there are so many people spending hard won resources reinventing the wheel and running up against the same failure modes.
Ha, this excerpt from the article is mostly unsupported by the report it links to (the report discusses challenges but has an overall positive tone):
That’s the damning conclusion of multiple studies published in the last few months. In June, the Turing Institute, the UK’s national center for data science and AI, put out a report summing up discussions at a series of workshops it held in late 2020. The clear consensus was that AI tools had made little, if any, impact in the fight against covid.
Though I suspect the main message of “AI didn’t help much” is well supported by other evidence.
To support the point about the importance of coordination: for COVID forecasting, a big success story in my opinion has been the Covid 19 forecast hub (led by my friend Nick Reich at UMass). His group basically systematizes public health needs from the CDC into prospective forecasting competitions with clear rules. There are dozens of models submitted (including lots of “AI models”, many bad, some good, and one developed by me 😉 ); they are combined into an ensemble model and consumed by CDC. So, there’s some great coordination.
How useful AI is in facing covid is debatable, but the measurement infrastructure underlying it all is not.
How quickly we had working global analysis up and running, seemingly from backyard analysts, was amazing.
AI (and the hype behind it) did a ton of that behind the scenes.
Better co-ordination is a nice goal, but that requires people finding each other in a lot of noise. Something that doesn't happen by itself. In fact, we probably rely on AI to do a lot of that too.
I guess the real problem is in our marketing and how the media perceives us. We shouldn't let number of failed models be a metric for success for our industry in general.
To answer your question @Beckett Sterner, we don't. We stack the failures on top of each other until we get something good and hope the hype train gets us there.
re: the coordination aspect, I think there have been a lot of initiatives aimed at just that, that have popped out recently. E.g., Wildlabs (AI for conservation), Climate Change AI (self-explanatory), ARIES (AI for env & sustinability), AI for the Planet, GoodAI (and it’s sub-communities), etc. I completely get the re-inventing the wheel, it definitely happens in conservation tech, where I mostly live. A lot is lacking and there is obviously much still to do. But just wanted to add some positive directions/projects to the conversation 🙂
This part of a recorded panel discussion on AI for climate in the private sector touches on some points made here on the value of coordination. I learned about it from this initiative/community that is likely of interest here too. @Gracie Ermi feel free to add this community to your updated resource list if it isn't on there already. (I can only view the original list.)
Hey all, as a corporate ML engineer, I have no real domain expertise, however @Emmanuel Dufourq and I are looking to do something impactful with Stellenbosch University. Perhaps some experts could give us an idea of something practical we could do.
Coming from a developing country, we really need to make a great case for funding, so it would be great if someone could share something that had real short term benefits. Thanks in advance, Henk
Wireless sensor networks/meshing! ie PAM devices, camera traps, etc. save us ecologists from hours of trekking to pull SD cards and give us the (semi) realtime data!😂 I work in Madagascar with AudioMoth & Swift’s. The benefits would be huge, on top of the saved time, personnel, supplies (& therefore cost), more real-time info could be used to act on “alerts” like gunshots or chainsaws. And would facilitate rapid censusing on the ecology/conservation side as well.
Transmitting even summary type stats would be great, via LoRa or other IoT solutions. With embedded ML becoming more and more used, you could load a detection model and then have daily summaries or just the images or sound clips that passed the model to limit data transfer. Emmanuel & I for example have an ML model to pull out lemur calls from the Madagascar soundscape.
I should also say I have no idea what the feasibility of all this is as I’m a conservationist/ecologist by training!
Rainforest Connection has a system like this with their Guardian devices, but it’s not (to my knowledge) open. The devices aren’t ‘for-sale’ like Audiomoths, SongMeters, Swift’s. But they have a wireless data transfer system that sends gunshot alerts to rangers on the ground. You could talk to the folks over there about those tech logistics I suppose
And what about also few-shot learning for bio-acoustics to help identify rare species with few data samples?
@Diego Marcos I'm working on that 🙂 paper soon with @Carly Batist but I just need to write write write... not enough hours in a day 😧
Oh nice! Looking forward to it! 😄
Thanks so much Carly! This is an excellent insight!
Just read this article by Hannah Kerner and 🔥🔥🔥
https://www.technologyreview.com/2020/08/18/1007196/ai-research-machine-learning-applications-problems-opinion/
I used to have real personal struggles with this as a phd student. I used to think that my work was not ever going to be as good as a theoretical paper and was boarder line going to give up on academia altogether because I was never going to be a LeCun or a Jürgen Schmidhuber. But turns out, application papers are super useful too!
And application papers spur new lines of theoretical inquiry.
@Hannah Kerner is here btw. If anyone wants to continue the conversation. Plus there is the machine meetup group on friday's that she and @Patrick Gray run. This week we are talking about very small object detection https://www.mdpi.com/2072-4292/12/15/2501
This work looks really cool. We were recently looking into tiny object detection in WAMI imagery and one of the biggest issues with modern network design we found is the aggressive early downsampling that occurs for compute reasons. I have not seen a lot of work exploring this information bottleneck, which is critical when your targets are only 2-10 pixels wide. I feel like there is some paper waiting out there where spatial attention, maybe even a transformer, provides efficient information pathways for the full res input data to inform later semantically rich stages of the network, but I havent seen it yet.
@Daniel Davila have you seen HRNet? -- https://www.microsoft.com/en-us/research/blog/high-resolution-network-a-universal-neural-architecture-for-visual-recognition/
It maintains a high-resolution pathway through the network for this purpose (not highest resolution, but you could easily change it). There is a really good implementation in the pytorch-image-models library - https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/hrnet.py.
It has been used to good effect in recent competition (e.g. winning solution in SpaceNet7) and has some good rankings in normal vision tasks (see paperswithcode). Actually one of the questions we discussed in the ML4RS reading group with the SpaceNet folks was "why do the winning solutions all upsample the imagery before putting it through their networks (e.g the winning solution upsamples 3x)?" I think the simple answer is because HRNet downsamples 4x and you need every pixel to distinguish building boundaries 😛
(You also might be interested in hard attention type approaches that don't "look" at the whole image https://proceedings.neurips.cc/paper/2019/file/8dd48d6a2e2cad213179a3992c0be53c-Paper.pdf)
Yep I sure have, thanks! As you point out, they also employ aggressive 4x early downsampling which you can either remove (which presents a practical problem for a lot of teams because you lose the benefit of imagenet pretraining unless you can swing that yourself) or upsample the input,. In both cases you take a large hit to runtime performance. Which becomes extra critical for real time or edge applications. I feel like there has to be a way to use context to attend to just very specific regions of in the input space at full resolution without shoving the entire full resolution image through the net
I'm also very curious about this! Hard attention methods seem to be the relevant research thread here, although I haven't looked too much into them. I think the idea is: run some convs, decide on a subset of patches to look at based on the feature representation, run more convs on just those locations. The just makes these methods hard to train (as you can't backpropagate through selecting a subset) but you get to save in computation at inference time. This is the other paper I'm interested in -- https://arxiv.org/pdf/2102.10212.pdf
@Chris Yeh has work using an RL approach to do something that sounds similar, though it's less about reducing computational cost and more about reducing $$$ spent on high-res satellite data: https://scholar.google.com/citations?view_op=view_citation&hl=en&user=6Omq2pcAAAAJ&citation_for_view=6Omq2pcAAAAJ:qjMakFHDy7sC
^Newer version of the same RL idea for choosing when/where to zoom: https://arxiv.org/abs/2003.00425.
Interesting new dataset for self-supervised pretraining: https://openreview.net/forum?id=BwzYI-KaHdr. They made the choice to include no humans, which is an improvement over a lot of the ethics/privacy concerns in Imagenet.
Dear all, we are developing an unsupervised video motion segmentation method that learns a persistent appearance representation for the moving objects, and are wondering if you may have videos of animals that you would like to get a segmentation out of.
It would be ideal if you tried running pre-trained methods like maskRCNN, but these don’t work due to out-of-domain generalization issues. If you have any collection of video we’d love to try our method and test it out! Please feel free to DM or email me (kanazawa@berkeley.edu). Thanks!
We have lots of video, not a ton of ground truth.
https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13011 I built a small tool that the wildlife video community uses on occasion.
Cool!! no ground truth necessary!!! we just want to run it on some videos and see!
cool i’m finding this! https://github.com/bw4sz/DeepMeerkat are the videos available?
yup. i've got many videos, i collected a few from other researchers. I also have a small user community from that tool. Its wildly outdated, but does the job for background subtraction -> motion detection.
The videos from that MEE paper are on google cloud, do you have a bucket? I think I can transfer to S3 or azure, but i'd have to check gsutil.
Hi @Angjoo Kanazawa would YouTube videos be a good source of data? We built an Intelligent Agent to datamine YouTube videos for whale shark sightings, but after determining if the video was appropriate (a simple NLP-based binary classifier), we extracted keyframes (based on detected whale shark confidence scores) pre-individual ID. Automated segmentation might be really interesting to help automate and improve the accuracy of individual ID.
Yes YouTube is totally fine! you can just point to a youtube video!!
Hi everyone 👋! DrivenData is looking for a Data Scientist and Senior Data Scientist to join our growing team or data folks committed to social impact! We run data challenges with a social impact, and do data consulting work as well. If you’re a motivated data scientist with a background in Python, come join us join us!
We’ve got a track record of engaging on conservation work including competitions (see here, here, and here), open source work, and building applications.
Learn more about the roles here: https://apply.workable.com/drivendata/
Greetings; we have a UK-based R&D job opportunity for developing deep learning applications for biodiversity monitoring (animal focused).
Specifics: Train NN models (based on the MMdetection framework) using the Amazon web services resources to detect animals from camera traps
Join a small team developing exciting applications “at the edge” and “on the cloud” to better service biodiversity monitoring, with a focus on animal welfare, providing affordable solutions to citizen scientists and specialists, as well as governments, local councils, and other organisations.
DynAikon is an R&D startup, developing IP, software and hardware. We have received funding from Innovate UK, Digital Catapult (Machine Intelligence Garage) and the EU (Horizon 2020). We are part of a current international consortium at the crossings of biodiversity monitoring and citizen science, with a focus on integrating AI and Machine Learning in the new services we develop. You will have access to this special network of international multi-disciplinary experts. More details: https://dynaikon.com/news/2021-11-12-job-listing/
Hi all,
I have trained a pose estimation network DeepLabCut to predict animal poses (key point coordinates). The next step would be to use these key points to predict the type of behaviour (Foraging, moving or other). I already labelled the data with behaviour. Which network would be suitable to transfer key points into a behaviour classification? Anyone who did this before? Are there any examples available?
Thank you!
Jorrit
Ive been playing around with the ResGCN work recently, I had some success with few-shot AR on a limited dataset of human activities.
https://github.com/yfsong0709/ResGCNv1
With such a low dimensional signal (keypoints extracted from the relatively high-dimensional video input), you also shouldnt discount the power of simple heuristic-based approaches. For example, if the deer's legs are straight it's probably standing, right? Etc... can be powerful information when trying to constrain the space of possible activities under observation.
Great @Daniel Davila. I will take a look at your suggestion. Although used on humans it might also work for Red deer. In your research, did your output also contained a behaviour classification or something else?
@Daniel do you know any examples of these heuristic-based approaches? Do you mean not using a DL-model but instead a machine learning algorithm. Or do you mean neither of them?
Yeah so for heuristic approaches, you dont need an ML model at all necessarily. You can reason about the relative locations of joints, particularly if you have a world model (e.g. you know which directions are up and down). For example, in humans, if somebody has their arm raised straight above their head, there are very few actions they could be doing (relative to the 1000s of actions possible in an open world).
And yes, I was using resgcn to take in a pose graph and output an activity (running, sitting, etc...)
Sounds interesting Daniel. Good to keep in mind that other methods could also work. Very happy with your tips!
very cool en useful development! @Jorrit van Gils, do you some more background information on your quest? Like, which types of behaviors do you distinguish, which one are expected to be very hard to distinguish from each other (where a sequence of images may help to improve the outcome)?
@John Beuving , maybe nice for you too, to have a look at
Hey @Jan Kees , thank you for the reply. As you can see I created three categories of behaviour ("Foraging", "Moving" and "Other") which I further divided into sub behaviours . Ideally you want to classify behaviour in detail and thus classify on the sub behaviours, however few images that most of these sub behaviour categories have makes it hard to properly train detecting these behaviours. Therefore I will first focus on classification of the 3 "main" behaviours that do have a fair amount of data and are more or less evenly distributed amongst the three groups.
that answer is already a year old.. Thx anyways 😉
Good luck to everyone applying for PhD programs! Hopefully we get a chance to collaborate in the future!
Check out this tiktok for computer science education week on Mission Unstoppable’s tiktok account! Featuring @Lily Xu - OLD ACCOUNT, DO NOT MESSAGE, @Sara Beery, Kakani Katija, Andre Perkins, and me - it’s all about how computer science is used in many ways to help the environment 🙂 https://www.tiktok.com/@cbsunstoppable/video/7039408911094697262?lang=en&is_copy_url=1&is_from_webapp=v1
Hello! I’m wondering if anyone is interested in talking to some elementary schoolers about yourself and your work? These students are learning how to code, and their teachers are looking to show them examples of real people who write code to tackle real world problems. You would only need to talk for 15-30 minutes (you can share as much or as little as you want!), and then the rest of the time can be for Q&A with the kids. When I talked to students at this school previously, they were extremely excited about the idea of building tech for wildlife, so I’m certain they would be very excited to hear from any of you! Here is a sign up sheet with dates, times and age groups for each session if you want to take a look! Let me know if you have any questions or want to talk though ideas for sharing your work with younger students! Thanks! https://docs.google.com/spreadsheets/d/1gN1IAKoNHc9_iJ_lIRFG9p22QNIZtePdkHdQ5D8LRpM/edit?usp=sharing
This is so cool!! Unfortunately I'll be at an interview those days, or I'd 100% sign up ❤️
Thanks for considering it anyway! I’ll make sure to let you know about similar opportunities in the future!
Hello wonderful people!
I am currently co-lecturing a Deep Learning Course in which students have to hand in a final coding project. I want to motivate them to work on a project on nature/wildlife conservation (or climate science).
For that purpose, I am setting up a list with resources: From open datasets and challenges, platforms and communities, publications to get inspired to tutorials and workshops.
Please feel invited to add to or comment on the list! Also let me know if you happen to have some smaller-scale project on which you could let some motivated Bachelor/Master students go wild on 🙃
The list is accessible here: https://hackmd.io/@Sp2mwyxuRAOrWv8BhDJ7eg/deep-learning-for-nature
Just a minor thing, but the ‘big’ directory originally came from that ‘smaller’ one, so it is all replicates, if you want to just reference the conservationtech.directory 🙂
you may consider some pointer to satellite datasets and challenges that have to do with land cover, deforestation, etc.
does someone already have some adresses for these? 🙂 thanks!
CropHarvest (dataset + code + benchmarks): https://github.com/nasaharvest/cropharvest
AI4FoodSecurityChallenge (dataset + code):
https://github.com/AI4EO/tum-planet-radearth-ai4food-challenge (competition is now closed but dataset + code can still be used)
Here is our open-source dataset on wildfire spread and associated weather, vegetation, and other features. https://zenodo.org/record/5636429#.YeWUzFjMKCc
Also, here is the paper from NeurIPS that describes the dataset. https://openreview.net/pdf?id=6nblryHxVbO
Hello everyone, I am devis and write from EPFL, switzerland! I am new here, but i am looking forward to engage in super discussions!
By coincidence I was reading this tonight. Super interested in the variable length representations through max/avg pooling of the features before learning a representation to concat. We have multiple (and variable) years of HSI data for tree species prediction. I had tried averaging the softmax scores across years for the same location, but it is not better than just taking the most recent year. I'm going to explore this idea in Fig 3. Thanks!
Hey Ben! happy to hear that you enjoyed reading the paper. Yes, in this case, depending on the aggregator you can explore from an average to a max to a learned combination.. back in the day I was working on clouds prediction on time series (using an RNN) and using a weighted combination ( with more weight on the most recent image) was working pretty well!
https://arxiv.org/abs/2012.10393
Somehow for multi-year problems I feel the domain shift issue is to be solved first, unless you are using separate models per year
AI LA challenge: remote sensing to detect invasive pests. $10,000 for first prize
AI LA is working with LA County to organize an Open Innovation Challenge to incentivize teams to find the most efficient solution using satellite imagery to detect invasive pests in LA County's tree canopy.
We will be hosting an information session on March 8th at 5 pm PST.
Apply here: https://lnkd.in/gCJgsziR
Read more on our blog: https://lnkd.in/g7KDUpCY
Hey everyone! Looking for advice for a friend. She works at a conservation NGO and has a technical but not CS background, and is looking to take a sabbatical and do a data science bootcamp for 1-2 months. Not intended to help her get a CS job, but intended to enable her to depend less on CS and data science people in her current role at the NGO. I know very little about these bootcamps, but some seem a bit predatory. Does anyone have any experience or advice?
What kind of CS skills are we talking here? Like AI, general programming skills, data structures?
I think data exploration and visualization in R and/or python, and maybe some data mining and rudimentary AI
DataCamp offer some great online course, I know a few people who have done Data Science and Python courses through them and were pretty happy with it. If she is looking for in person courses then my recommendation would be to see if any university nearby is offering these over the summer (Quite often they have partner institutes which run private short courses for them - and thus they are normally of a higher quality).
Hi Sara, I know this isn’t exactly what your friend is looking for, but if she can take a bit longer off, UCSB’s Bren School offers a great 1-year masters in environmental data science that really emphasizes programming and data science for environmental professionals. Most of the students are exactly like your friend--environmental professionals who want a better understanding of these quantitative skills. https://bren.ucsb.edu/masters-programs/master-environmental-data-science
One of my colleagues recommended a specialization course offered by Johns Hopkins. This one is R heavy apparently
A lot of universities are offering MOOCs on a revolving basis now, not just tied to academic calendar. These courses could be a good way to get your feet wet in a topic without getting scammed
Hi Sara, I don't have experience with these boot camps, but I would be cautious about paying for this type of service. There are a ton of free (or cheap) resources for learning CS concepts. It may be worthwhile to look at some of the resources collected here to see if your friend would benefit from them: https://github.com/Developer-Y/cs-video-courses#introduction-to-computer-science
If they know anyone who can help explain specific concepts, that would be ideal!
I agree that there are a lot of resources available for free, but I also understand the interest in a guided learning experience with access to mentorship!
Yeah, the mentorship piece is very Important. Though, it might be a good idea to get their feet wet with basic Python or R before paying for instruction. I usually recommend the sentdex videos on youtube to students new to coding/Python. It seems to get our summer students up to speed quickly: https://www.youtube.com/playlist?list=PLQVvvaa0QuDeAams7fkdcwOGBpGdHpXln
Hope this helps! 🙂
Hi! I did an MSc. In Computational Biology in France. But for me it wasn't enough. During this MSc. I tried a Data Science Bootcamp (Le Wagon). That wasn't cheap at all but for me it worth the money. I needed a structure to launch myself into this adventure. Even if the resources are available for free on the internet, the expertise they provide and the support (intellectual and emotional) they give me were very helpful. It allowed me to integrate an internship in Deep Learning (that ends in July) for bird conservation and I hope to continue with a PhD in September. I hope this little testimony helped! 🙂
Might be of interest 😉
Pl@ntNet-300K: a new dataset to improve plant images identifications https://cos4cloud-eosc.eu/blog/plntnet-300k-a-new-dataset-to-improve-plant-images-identifications/
I'm defending my PhD thesis this Friday at 9am PT if any of you want to call in! you can attend via zoom at https://caltech.zoom.us/j/89103003020?pwd=Q-jR9GiObmTAS59G462SokuPM494Ph.1
Best of luck, @Sara Beery. Recently I defended mine and I passed, doing post-examination changes to Thesis now. I hope you will pass with flying colours. 👍
Wow! my paper is invited to CV4Animals at the poster session! 💃
maybe yes! I am waiting to know which presentation modality I am allowed to choose. 😄
Hi all! does anyone know of an open resource for satellite/aerial animal tracking data?
I know he has been working on this in Namibia :)
we were working with some collar data back in the day with the Wildlife ecology group of Wageningen university. But there was no sat / aerial image. Otherwise we must have some drone videos lying around, but they are not labeled
Gotcha! I was mostly remembering hearing about drone flights
Like from tags? Movebank, MegaMove, etc. Or do you mean from drone & satellite imagery itself? Are you looking for datasets or software to analyze said data?
thank you! yes, i am looking primarily for drone video-based datasets (and open to learning about any/all related resources!)
The Conservation Tech Directory has some I believe - e.g., https://geonadir.com/. @Elizabeth Bondi also has the BIRDSAI dataset - https://lila.science/datasets/conservationdrones. And of course Sara already mentioned Ben - his AIDE project (https://github.com/Microsoft/aerial_wildlife_detection). And there’s another one I know of on LILA BC for seals - https://github.com/microsoft/arcticseals
Thanks Carly! Let me know if you have questions, Felipe
We have some drone footage of kenyan ungulates and some other miscellaneous footage. Feel free to get in touch if interested.
FYI @Felipe Parodi - just came across this ML model to detect elephants from aerial imagery: https://www.arcgis.com/home/item.html?id=4976292298c440e686aa339e52da2dbb / https://doc.arcgis.com/en/pretrained-models/latest/imagery/introduction-to-elephant-detection.htm
Hi everyone! I’m currently looking for a bit of extra funding for my PhD research. Does anyone have any experience applying for Rufford Small Grants? If not would anyone have any recommendations for ‘small’ pots of money (£2,000-£6,000) for a super cool project on bird trade, technology and extinction? 🙏
I review a lot of rufford grants
I’ve applied to Rufford before but have been unsuccessful haha.
Other small conservation grants I know of: Society for Conservation Biology, American Philosophical Society’s Lewis & Clark Fund (if you’re doing fieldwork), Explorer’s club student grants.
And a lot of zoo ones, though idk how the pandemic has affected the availability of those. Sunset Zoo’s conservation scholar program, Columbus Zoo’s fund for conservation, John Ball Zoo’s wildlife conservation fund, Chicago Zoo’s CBOT endangered species fund, Phoenix Zoo’s conservation program, Oklahoma City Zoo’s conservation action now grants, Cleveland Zoo’s future for wildlife grants, Greenville Zoo’s conservation grants, Roger Williams Zoo’s Sophie Danforth conservation fund, Riverbanks Zoo’s Satch Krantz conservation fund, Fresno Chaffee Zoo’s wildlife conservation fund
Also, Cornell Lab of O has a directory of bird-specific conservation grants - https://www.birds.cornell.edu/landtrust/bird-focused-funding-opportunities/. Ornithology Exchange has a similar one - https://ornithologyexchange.org/funding/grants/.
Thank you so much I’m applying from the UK so will check eligibility for some of those
Ah ok yeah not sure on eligibility, I think at least some are open to internationals
Hi everyone,
I am currently looking for a potential mentor from a non-academic organization that focuses on animal movement research for my NSF INTERN fellowship application. I would like to know what are some non-academic organizations study animal movement (mainly quantitative research). Also, are there particular people I should consider reaching out to? I am a Ph.D. candidate from the John Fieberg and Allison Shaw labs at the U of Minnesota.
The only non-academic organization that I can think of so far that studies animal movement is WWF. Could anyone give me a list of NGOs and companies (+people) that study animal movement analysis?
Thanks!
Hi Dennis! My institution (Monterey Bay Aquarium Research Institute) would qualify under the INTERN program. If you're interested in anything ocean, I'm happy to discuss.
Thank you, Kakani Katija for letting me know! + that MBARI is qualified under the INTERN program! I applied to the program a few months ago with MPIAB (Max Planck Institute of Animal Behavior) unfortunately, NSF considers MPIAB as an academic institute and said no. I will definitely reach out to you once I discuss it further with my advisors! Thanks again!
Check out the conservation tech directory! Tons of organizations listed there. ‘Organization’ is a resource type you can filter by. And then use keywords like telemetry, movement, etc. https://conservationtech.directory/
Hello Dennis! Do you know if bio-inspired design organizations would work? In terms of animal biomechanics I can think of a bunch, just need to k now what type of bio-movement you are focused on (e.g. movement in fluids, movement on land, etc.)
Hi Andrew, thanks for reaching out! - Most of my work so far focuses on telemetry data on terrestrial animals - however, I have dealt with some aquatic species and currently working on the arctic terns migration with wind fluids - but I have not yet dealt a lot with the fluid movement compared to the terrestrial animal movement on land - but I am open to all the suggestions!
Hello,
My name is Deva, I'm french and I'm currently an intern (last year of master in Bioinformatics) in the implementation of bird recognition by camera trap (MobilenetV1 and YOLOv2). I want more than anything to continue in this field (not necessarily on the birds). I am therefore looking for an opportunity to gain knowledge and skills outside France (Canada, Australia, Scandinavia) for September 2022 with a PhD or an engineer job. I'm leaving you my CV and I'm ready to receive all your advice.
Thanks a lot!
Hi, I’m looking for experts in insects and computer vision. I have a journal paper and need a reviewer for studying pollinator monitoring, it counts, tracks and analyzes insect behaviors like honeybees. if you have any suggestions or interests please DM or respond to this thank you!!!
I've completed a short video to explain how we data science to boost nature conservation. Comments, ideas and help are welcome !
https://lnkd.in/dsttNrRf
Hello everyone! I am trying to find some software which would set a drone to take images of forest canopies at randomly sampled subplots. I really only know about drone photography, so this is a new front for me. Any pointers would be very welcomed!
Hi Jinsu - you may want to pop this question on wildlabs and tag it into the drones group. Lots of folk there using drones for forest monitoring, they'll be able to give advice.
What kind of drone do you have? There are some pretty nice apps out there that allow you to set geofences, waypoints, and even detailed sequences like "fly here, point camera at this gps coordinate, take picture, move on..." most of the big players support these out of the box, and there are 3rd party apps (usually pretty cheap) that can exploit nicer payload navigation and control systems to fly all sorts of missions.
Hi @Jinsu Elhance we have been using dronelink to fly dji systems and have found it to be very stable. Maybe it would suit you?
does anyone have experience loading multi-dimensional arrays (more than 3 channels) into pytorch at scale? We got lots of GPUs, but have been using rasterio in our dataloaders and it scales very poorly. GPUs are often starved and waiting. Save as numpy .npy files? For training its not critical, but during prediction (10 to 100 million scale), we really need to focus on improved performance. Lots of solutions out there for 3 band data (https://developer.nvidia.com/dali). python's GIL seems to really limit rasterio performance, multiprocessing/thread provides no clear benefit. Dask?
Hi Ben, I imagine you are working with satellite images. In my experience saving them as into hdf5 datasets brought a nice performance gain especially when scaling up to larger applications. Might also be worth trying to evaluate which part of the data loading process is the bottleneck to try to optimise the implementation of the data loader itself.
thanks. the bottleneck is loading, its all been profiled. So you make a single h5 object and each step in the dataloader loads an image. Do you use multiple workers or load on the main thread?
In the case of Sentinel 2 you can for example save a h5 file for each tile/date combination, and that file can include a dataset of all the patches that you will want to extract from that tile so that will be of size: (#patches, height , width , #channels). Then you open that file the first time you want to access it (in the getitem() and not the init) such that you can use multiple workers and each has it’s own connection, also you need to make sure the h5file is closed when you delete the instance of the dataloader. I hope that helps!
Riccardo's solution has worked well for me in the past as well. I would also add that you can make use of data loader parameters such as memory pining, num_workers, and prefetch factor. Also, if your machine has many CPUs, it may benefit from limiting the number of threads that torch uses: https://pytorch.org/docs/stable/generated/torch.set_num_threads.html
Something reasonable like 4 workers with a prefetch of 4 usually works for me, but you can tune this.
thanks @Alex Brace, very familiar with this tools in other contexts, they provide no benefit in this case. @Riccardo de Lutio I'm interested in why you open it in the getitem and not the init. I see your point about the multiple workers. I would have thought that the cost of opening the .h5 would swamp that benefit. I'm going to try the same logic with rasterio. Thanks everyone for the feedback. I appreciate it.
The h5 needs to be opened in the getitem so each of the data worker threads can get their own handle. The init is run only by the main process I believe. I normally implement this by setting a boolean member variable in the init isopen=False . Then in the destructor you can have something like if self.isopen: self.h5_file.close()
This way you only open the h5 file once in the getitem e.g.
if not self.is_open:
self.h5_file = h5py.File(data_path, "r")
Also if you happen to be on a cluster that supports an efficient node local temp storage option, then you can get additional benefit by copying your h5 file to the tmp space and passing the node local path to your data loader. This will minimize file system I/O. You could do this in your main function for instance (it is essentially an efficient cache you can use while your job runs).
h5 also supports seamless compression and nice chunking features that you may be able to make use of. I don't know the specific details of your dataset/application but it's something you may consider if the dataset is very large.
Hey Ben, do you have the code you are using anywhere? Simply reading GeoTIFF files with rasterio is usually quite fast for us
You can get large slowdowns if you are doing any reprojection steps with the data using rasterio, or trying to do windowed reading with a non-tiled GeoTIFF
here is the dataloader that reads .tif files from disk using rasterio. https://github.com/weecology/DeepTreeAttention/blob/1b704c74752c710bc03d0728f847be902f915d05/src/data.py#L240
<a href="https://github.com/weecology/DeepTreeAttention">weecology/DeepTreeAttention</a>
all of the profiled time happens in rasterio.read()
or, rio.open().read() https://github.com/weecology/DeepTreeAttention/blob/1b704c74752c710bc03d0728f847be902f915d05/src/utils.py#L60
<a href="https://github.com/weecology/DeepTreeAttention">weecology/DeepTreeAttention</a>
atleast according the snakeviz and cProfile.
is it easy to post a link to one of the image tiffs?
increasing num_workers > 0 makes things much slower. The only solution I have found is to preload everything to memory. That works during train/test, but during prediction we are talking about 10> TB. So i've developed an on-the-fly cropper to cut out the crops (tree crowns) and predict them. There are some other complications, but that's the crux of it.
thanks @Alex Brace, i'm definitely going to try the local storage idea. IO seems to be the crux.
just as a sanity check, I'm seeing that file is all 0's, is that suppossed to be the case?
As this data isn't georeferenced, and you don't need a dataset object that does windowed reading at all, you could try to use the tifffile package -- it calls into libtiff instead of libgdal and I've seen it is generally faster for this use case
sorry, that was a coincidence, its actually a multi-temporal model, and if there isn't any data for that year it gets padded with 0's (currently working on not reading the padded data). I'll have a go at tiffle and all these suggestions, thanks.
Im not sure how fast it is, but if you are loading large cubes just to throw some channels away a change of format may help. I used this package called spectral (https://pypi.org/project/spectral/) to work with large hyperspectral imagery in the past. Might be what you're looking for. I liked it because it let me load just the channels I wanted from the data cube for supported formats
Also, if loading larger samples, are you checking the simple stuff like that your dataloader isnt maxing out your system memory and dipping int swap or anything?
If you are spatially cropping a lot of individual images, COG comes to mind as well (https://medium.com/spatial-data-science/cloud-optimized-geotiff-cog-with-python-8e0a22b884c5) . We have implemented workflows that preprocesses all our source imagery into COG first, then we can rapidly sample sub regions across many large images quickly (like sticking together a big video tubelet from an ROI across time)
If you cache this cog preproc, you can reuse each training/inference run so you dont have to recompute each time
@Jonathan Crall has a package that makes this pretty nifty, if it helps https://pypi.org/project/ndsampler/
I don't know if its useful, but if anyone ever comes back to this thread, I found a couple improvements. 1) using the scratch space for large rasters is useful, but not huge, 2) For reasons that are still unclear (see https://github.com/microsoft/torchgeo/discussions/594, https://github.com/rasterio/rasterio/issues/2053), keeping rasterio objects open blocks mulit-processing for pytorch's dataloader. While it feels counter-intuitive, opening and closing the handle in the getitem is the key. This allows you to bring the needed CPU resources to bare.
<a href="https://github.com/microsoft/torchgeo">microsoft/torchgeo</a>
<a href="https://github.com/rasterio/rasterio">rasterio/rasterio</a>
Thanks Ben, I don't think we've tried that in torchgeo, will have to see what happens there
@Kakani Katija featured in Computer Vision news after CVPR!! https://www.rsipvision.com/ComputerVisionNews-2022July/18/
Random question (hence why random channel 😉 ) but does anyone here donate to charity each month or on a one-off basis? I would be keen to chat about how you do it/where you find good environmental causes etc.
React with a 🎗️(ribbon) if you do!
I enjoyed this Sam Harris podcast episode with William MacAskill. Check out https://www.givingwhatwecan.org/.
William MacAskill is an Associate Professor in Philosophy and Research Fellow at the Global Priorities Institute, University of Oxford. He is one of the primary voices in a philanthropic movement known as “effective altruism” and the co-founder of three non-profits based on effective altruist principles: Giving What We Can, 80,000 Hours, and the Centre for Effective Altruism. He is also the Director of the Forethought Foundation for Global Priorities Research and the author of Doing Good Better: Effective Altruism and a Radical New Way to Make a Difference.
@Declan thanks for sending this over - awesome set of links and I will do some reading!
Do you use Merlin? 🐦
Our research team (Sunnie Kim, Elizabeth Watkins, Olga Russakovsky, Ruth Fong, Andrés Monroy-Hernández) is conducting research on understanding explainability needs of users of real-world computer vision applications, and how existing explainable AI (XAI) methods satisfy or not satisfy the identified needs. We're looking to interview users of the Merlin Bird ID application who regularly use Photo ID or Sound ID. If you are interested in chatting with us for about an hour over zoom, please fill out this short google form. Study participants will be compensated with a $30 gift card or a $30 donation to a bird conservation organization. Thanks and happy to answer questions via DM/email!
Hey sunnie - this might be one to pop into the AI for conservation and acoustic groups on wildlabs https://wildlabs.net/
Thanks so much for the suggestion!
Hi Sunnie, how frequent of a Merlin user does one have to be to participate?
Hi Carly, we're trying to talk to a wide range of users, so please fill out the form if you're interested! Roughly speaking, we're looking for users who have used Photo/Sound ID over 10 times in total and at least once in the past month?
Does anyone have an opinion on whether we can use the same holdout dataset to evaluate AND calibrate the model? I'm working from https://arxiv.org/pdf/1904.01685.pdf and they are vague on if we need a third dataset (train/test/calibrate) to assess temperature. Sure, having more holdout data is always awesome, but given that almost never happens (most projects barely have enough test data), what bias is introduced by calibrating on the test dataset? My tree species model is typically overconfident.
I always try to have one and also th ecomposition of the test set is super important to avoid data leakages (but oyu know that better than me). In stat, when sample size is small, people sometimes train models with one test sample out and predict it, then train another for another test sample, etc. It’s a bit insane computationally but…
For aerial images, I always try to have the sets geographically disconencted, to avoid spatial biases of structures seen in train leaking in test (which would happen for example if you sample your test samples randomly from the images you have. Also, normalisation strategies induce biases if you normalise on the whole data (so you use the stat of the test set to normalise). In EO it changes results massively (= lower results, but honest) to split/normalise right.
...Picking up this older discussion, as it looks important! My philosophy on this is that we should always do our best to replicate the expected application during testing. Seems to me that calibration could be done on the training data...I'm curious about thoughts on this?? Here's an example of data leakage- what happens when a random split of objects is used for training/test splits compared to when parent images (i.e., not just objects) are separated for training/test splits.
Hello there 👋.
Just curious to know if anyone in this channel is attending the IUCN Africa Protected Areas Congress (APAC) in Rwanda?
this sounds super cool - I wish!
Has anyone tried out Google's Vertex AI Platform and have any opinions on it?
https://cloud.google.com/vertex-ai
hi all, we've got a highly challenging vacancy for a full stack developer to work on highly advanced biodiversity monitoring solutions.
Interested? Let me know!
Ps Candidates need to have a permit to work in Europe
Many orgs don't start off with enough funding for full salaries but instead need contract labor to get initial projects off the ground, which is Wild Me's experience when we started. I am just passing along a recommendation in case any of your teams need contract work for Python or DevOps. Wild Me just finished a two-year contracting engagement with Mulich LLC, which is really two experienced software professionals (Michael Mulich and Karen Chan). They were embedded with the team and are clearly very experienced Python hands. They handled the high complexity of our platforms with ease, including suggesting good designs and debugging complex and thorny issues. We will continue to work with them on future projects, but they're a very good resource if you need skilled Python or DevOps help but don't have a full position funded. They primarily work with nonprofits. They are not AI experts but rather can help build the scaffolding around AI-focused projects. Michael is the primary contact: michael.mulich@gmail.com
I am posting this with their permission, and I am independent and have no business interest in their work.
I second this recommendation, they are fantastic!
I am. I'm just transitioning careers to this field and would love to connect with folks from AI for Conservation!
@Leonardo Viotti yayy!! would love to meet if you have availability 😄 was also hoping to connect in person with the cool people in this group 🙂
Maybe you guys can coordinate an AI for Conservation happy hour?
That sounds like an excellent idea! Would love that!
Ditto the entire previous thread (about ESA) for TWS in November. I know it's a long way off, but I'm a compulsive agenda-planner, so if folks from this community will be there, let me know and we'll see whether we have critical mass for an informal event, or just want to do random 1:1 greetings, etc.
@Matt Weldy and I were just discussing this!! We're going to plan a happy hour there too, and I would love to see more of these community meetups at conferences going forward 🙂
I’ll most likely be at TWS this fall as well!
I'll be there Monday mid-day through Thursday mid-day... so I semi-arbitrarily provide one vote for Wednesday evening for an AI for Conservation hangout/dinner/etc.
I'd also like to organize at least one early-morning, rain-or-shine run, maybe somewhere with a moderate p(wildlife sighting). I'll semi-arbitrarily nominate Wednesday morning for that.
Both sound great. It would be really nice to meet others from this group.
Wild Me should be hosting a booth there and attending the AI workshop. I think we missed the window to propose a talk, so we'll be there to applaud presenters enthusiastically. We'll definitely be at the happy hour.
I will also be at TWS and would be happy to help plan a happy hour!
@Sara Beery @Matt Weldy Did y'all end up picking a time and place for an AI4C meetup at TWS?
We got AI for Conservation and Friends listed on the calendar for Wednesday night. Although it looks like the event organizers listed the location as 'offsite' instead of listing Black Label Brewing [https://www.blacklabelbrewing.com/]. I'm looking forward to meeting people!
Also if "AI for Conservation" is not sufficiently niche for anyone, as per above, if there are folks in this group who will also be at TWS and also like to explore new places via running, I'm planning to drive to a trail head for a morning run on Wednesday morning, leaving the hotel area around 6. I'm committed to finding a moose. On Tuesday and Thursday around 7 I'll do a local run from the hotel area. Mail me if you're interested!
I have a question that I think will at least entertain this community, but is sufficiently unanswerable that it definitely belongs in the "random" channel... how many ecologists are there? Like in the world? I mean that exactly as literally as it sounds. Have at it.
ESA has ~9000 members, but that’s obviously a minimum considering there are many more that are not members. I guess it depends on what you define as an ecologist - people may also identify as environmental scientists, wildlife biologists, etc. The US Bureau of Labor Statistics has categories for environmental specialists (87,000), wildlife biologists (19,000), and conservation scientists (39,000), and across those 3, there are >130,000 jobs per the 2020 data, though not everyone within those categories is likely to be doing ecology by the strict definition of the word.
Oooh, those are good numbers. I'll take that as a rough count for various definitions within the US; I wonder if we have any way to anecdotally extrapolate that? Even for this semi-arbitrary exercise, scaling based on population seems dicey. Is there any conference, for example, that we for some reason believe has geographically proportional participation that we could use to extrapolate those US numbers? Or maybe somewhere there's an estimate of the number of degree-granting programs in various departments by country.
Hi there, is anyone attending ECCB (European Congress of Conservation Biology) next week?
We (@Gracie Ermi & I) just passed the 1-year anniversary of the Conservation Tech Directory launch! And published a new update to commemorate, if you’d like to check it out: https://conservationtech.directory/. While you’re there, you can subscribe to our listserv on the site, to hear about future updates & get these snazzy emails every other month.😊
Thanks especially to the many of you who have contributed resources yourselves over the year as well - we so appreciate it! 🤩
Figshare: https://doi.org/10.6084/m9.figshare.15442200.v15
Form to add new resources: https://docs.google.com/forms/d/e/1FAIpQLScfbapq1sQdVWzEGVR1ezR2_TkKEDcV9Bcp-3idV6ZmmJxpHw/viewform
this is a fantastic service, i've recommended so many students here who want to get into AI for conservation!!! thank you Gracie and Carly :)
Out of curiosity, for those who collect video/lots of digital data in the field, what data storage/management devices/software/systems do folks use or suggest (or have any kind of advice)?
We currently generate about 5TB of data per trip (~2wks) collected across many devices (robots, GoPros, etc., maybe 500GB per device per day), but are likely going to scale up (~5x) pretty easily, in areas with very limited internet and rough roads. We currently travel around with a Synology NAS/RAID setup and do a very annoying data xfer of each device onto it at the end of each day (crazy mess of ethernet cables, routers/switches, laptops, devices, etc.). The alternative solutions has been 1/2TB NVME drives, but even that has many pain points. Thanks so much in advance!
Hi @Levi Cai,
the last three years I collected and analysed beyond 100TB of videos (multiple years of footage) each year and had the need to store them for processing, occasional randomly access and archive them.
I used cloud providers (spending a lot of $) and hosting everything on premiss in our office.
I tried to describe our setups a bit, but there is so much that this is incomplete and might look chaotic. we can have a call if you would like to have more insights or we can organize a small support group if more are interested.
In generally, I try it try to think of different tiers of storage (similar to the cloud providers) and replicated the same locally:
- deep archive (accessed << 1 per year and in batches)
- archive (accessed a few times per year)
- processing (fast access with low latency)
Each study is only in one of these steps and therefore i can schedule data transfers accordingly
But your approach may be different, depending on your use case.
In the cloud (e.g. AWS):
- Get a snowball / snowcone from AWS for uploading the data or access to the backbone of the university (100Gbit/s).
- S3 storage (One Zone-Infrequent Access) for holding the data while its processed. (directly upload to the class to save money.
- EBS storage for processing (copy a folder, bucket that should be analysed to it).
- S3 Glacier / Deep glacier for archive.
OnPrem:
- USB 3.0 SSD to collect the data (I got ~100 PNY Elite USB 3.0 Portable) -> Main idea is to ignore transfere times to the server by having enough on hand to alway get a fresh SSD
- NAS with ZFS pool with a lot of spinning disks and IO bandwith for holding the data while its processed or not ready for archiving (it’s faster than an ssd [throughput], but has a greater latency than an ssd).
- NAS with ZFS pool with a few disk, great deal of redundancy and lot of storage. Archiving (disk are mostly turned of / idle state, since they are no access)
- NVME and Ram disk holding a few jobs worth of data from the NAS for processing and low latency access on a workstation.
- Workstations and NAS are connected via 10G ethernet and proper switches.
- Lastly (we are not using it) tape drive for real archiving.
Regarding hardware choices for our DIY approach:
- Modern mainboard (10G ethernet), cpu, a good amount of ram.
- multiple JBODs enclosures from ebay (just a bunch of drives)
- PCIE SAS / fiber controler card to interface with the JBODs
- PCIE USB cards, such that you can attach a lot of external storage for transfere
- True NAS scale as OS
- a couple of SSDs (nvme) as caches.
=> Since cost is always factor, the server cost me about 500 Euro (mainboard, cpu, ..), 200 Euro per 15 hdd slots (JBOD, …) and then your HDD price.
Painfull learnings:
- Mailing your data to AWS (snowcone) might be faster than uploading them.
- If you want to utilize a 10G or 100G uplink, make sure your hardware can handle it.
- Proper cooling for the HDDs, they dont like heat.
- Never store single images on a HDD, accessing them becomes a pain when you want to access all of them. Ingesting them as .tar in batches helps.
- Be really care full, if you do not have a backup with rm. Having a lot of data might result in not being able to afford a backup.
- Depending on the size of your zfs pool, you might need to buy 5 additional 16TB drives if you want to expand it (costing you 1.500) size your pools wisely (e.g. small for archive (big hdd), large for processing (small hdd).
- Have a good system for labeling, tracing and checking external storage media (know where it is, its orgin, next step and their checksum). Loosing or not knowing the origin of data is bad 🙈
Regarding the mess of switches, … I bought a server cabinet of the second hand market for 50 Euro and put everything in there. Event the data ingestion stage, so only it is messy. Additionally i have a small shelf for external HDD, … that i used in the past.
In general we stopped to transfer anything on the field, since we always did mistakes on the field an stuff gets lost.
We just replaced recording storage devices and focuses more on how to process the huge amount of data
Wow thanks so much for the sharing!! I’ll read through everything before asking more details.
In general we follow vaguely similar philosophies and setup, but you seem much more organized about each portion.
Late to the party — for on prem, did you consider something like a minio setup over the zfs pool?
@Frederic thank you, that’s a good summary of the hardware and data management considerations - curious about the wattage required for the setup you have (we have ‘in the field’ considerations too).
@Howard L Frederick sadly I can not give you any numbers on the wattage user by the setup. In only measure our entire server cabinets (incl. GPUs, …). The share of the NAS is probably 200 - 450 watt under load, but i have not measured it (Ryzen based NAS, 2x JBOD with 15 HDDs each).
I know that you can bring it down with a lot of optimisations to ridiculously low numbers in IDLE, but out of the box it will draw a lot.
If you want to reduce the energy used, buying second hand might not be the best option.
I want to reduce the energy consumption on my home NAS and got across this useful video for optimisation:
https://youtu.be/MucGkPUMjNo
#tech4wildlife week is coming back!
https://wildlabs.net/community-announcement/next-week-join-7th-annual-tech4wildlife-photo-challenge
Nice story and video about our last data collection campaign in Svalbard: https://www.nioz.nl/en/news/using-eyes-in-the-sky-to-locate-seals-in-a-rapidly-changing-arctic
Awesome! You are probably aware, but Erin Moreland at NOAA is working on similar images and species, also @Casey Youngflesh mentioned walrus. If there are data eventually making it, this would be a nice addition to airborne data collection. Happy to email anyone if looking for more data/cross-training.
Really great, @Devis Tuia! At some point would love to connect with you about this work (cc: @Benjamin Kellenberger). Some definite connections with walrus-related work I’m doing!
With pleasure @Casey Youngflesh. I am sure Jeroen (the Phd candidate working on the project) will also be delighted to sit together with e-coffee!
Listen to Tanya Berger-Wolf's conversation about AI for Wildlife Conservation and imageomics with Lauren Burke on the Women In Analytics After Hours Podcast. https://www.wildlabs.net/link/wia-after-hours-episode-5-ai-wildlife-conservation-and-imageomics-dr-tanya-berger-wolf
Looking at the awardees, clearly USFWS got excited by AI and went all in on the idea AI = genius in this funding round 😂 https://wildlabs.net/article/us-fish-and-wildlife-service-announces-theodore-roosevelt-genius-prize-winners
Sharing here as there are a heap of relevant AI things this issue!
Our latest WILDLABS Community digest has just hit inboxes! Don't have time to keep up with everything that's happening in the conservation tech world? This email is for you! It's designed to be an easy-to-scan summary of the latest conversations, projects, research, events, jobs, and funding opportunities that are happening in conservation tech right now. Delivered fortnightly, straight to your inbox. Check it out: https://mailchi.mp/wildlabs/digest-23-september-2022
Hi everyone, I was wondering if you knew of any interesting AI for Conservation projects in New-Zealand and people working on them?
Thanks a lot! 🙂
Would love to know, and in Aus as well
@Victor Anton has a lot of projects there through https://www.wildlife.ai/
And in Aus there are tons! CSIRO and Data61 do a lot of work in this space, one example is this COTS detection project: https://algorithm.data61.csiro.au/how-were-using-machine-learning-to-detect-coral-eating-cots/
You might try searching/filtering by ‘australia’, ‘new zealand’ or related in the Conservation Tech Directory!
That's super useful thank you!
@Déva Sou As Sara pointed out Wildlife.ai is involved in a few projects and we recently helped write the "AI for Environment Aotearoa report". There is also an active "AI Alliance for Biodiversity" group coming out of the Eco-index programme. I would be happy to provide more information about the AI/environment community in NZ. What's your interest?
@Sachith Seneviratne adding to the list of Aussie projects. Xylo Systems have an open source list of Conservation Collaborators and funding
That is neat Victor, been meaning to follow up with some potential Aus collabs in this space, but there's always other work to do haha.
Not sure if this is too far off tangent from the group's focus, but is there potential to use SD finetuned end-to-end as a source of data augmentation in some of the cool models/datasets I'm seeing on this slack?
Attached couple of images for the prompt "A close up painting of Sachith riding a dragon", theoretically it should be possible to explore as a data augmentation strategy for rare species perhaps? This is finetuned end to end on 5 images of myself on an A100. Throughput is about 20k images per week on a P100, on an A100 it's about 120k per week or so. Apologies if this is silly. I mostly use this for urban design/computer vision work, but thought it may be of interest.
Combined with https://makeavideo.studio/ we may have full length custom movies soon? DALLEWOOD 😄
I personally believe these models are maybe one or two conference cycles away from being ready to solve data curation for many domains. It's really amazing the progress being made with these LFM. Super exciting stuff. I skimmed a paper on arxiv the other day (cant recall if published or find link now) where someone was using one of these to produce synthetic brain scan images for training ct models
Yes I saw that as well. I think that was a diffusion model and not a text to image diffusion model, but since it's not an interesting domain for myself I only skimmed it. There's a couple of cool ideas on ICLR this year: https://arxiv.org/pdf/2209.14988.pdf
Not silly at all!! We've been taking about this at Visipedia. For now those models don't handle fine grained categories like species very well, but I can definitely imagine they'll improve on that regard
@Sara Beery would you be able to share a few such species with me? I'd be interested to see if I can generate anything interesting. Did you finetune the whole model end-to-end? Presumably if it's for a single class you don't mind a bit of textual drift, I think you'd be able to maintain enough of the latent space to generate novel renditions of your target category.
Understand if you can't share details as well.
No no, nothing that deep! I was just using off the shelf models with common specied names, and they were often wrong in strange ways. Things like putting tails on bobcats or changing the face markings on birds
Oh, this is fine-tuned end to end
I thought it could be an interesting way of handling data imbalance for FGVC problems
well if you can share a few good 'things' to generate I'm happy to follow through and we can maybe discuss if I actually manage to get any good results.
I would maybe start with bird species since there are good field guides around the differences between them and just first try to understand when these models are capturing those important details and when they aren't
OK stanford birds maybe? is that the name of the dataset?
I'll dig through and have a look.
@Sachith Seneviratne this is something i've been playing around with as well, and while there are some details to be worked out I agree that this could be an extremely promising direction. Excited to see where your exploration takes you, and i think this probably deserves a dedicated session / channel to explore this idea 👍
Certainly, I've cleared up my projects for the next 2 years to make plenty of time to understand/do some work in this space (haven't decided theoretical or applied). I've been following since the avacado armchair days when DALLE was announced. I wanted to but missed out on miniDalle Jax week because I had a deadline that week. I'm still trying to figure out how drift affects generalizability of renditions in end to end finetuning, I've been trying to find hard test cases, the birds are a great place to start.
OK I've run into my first set of issues with generating a few of these classes, this is perfect. @Sara Beery or anyone else let me know if you're interested in continuing exploration or contributing to this work, probably easier through email : sachith.seneviratne@unimelb.edu.au. I'll share the animal related result to the slack once I've solved the problems, otherwise.
Might be related but I'm running this in an inpainting setting for more non-art real images (plumage.ai) and will start looking at dreambooth for tuned versions as well (once i get my gpu). There might be room to selectively ablate parts of camera trap images and inpaint for data augmentation.
For example, here's a tattoo of a dog that doesn't exist
You bought an A100 to finetune this? that's pretty dedicated haha
I'm not convinced that inpainting implementations on this are really 'inpainting' in the DALLE sense, but I haven't dug into inpainting too much.
@Abhay their tattoo results look interesting, I might dig through when I have a bit of time. Do they have a paper or a github?
Sorry just saw the earlier texts. I'm getting a 3090ti as an upgrade over my 980ti for this and other purposes (which is sufficient for dreambooth apparently). As for the tattoos, this is the standard inpainting pipeline from huggingface diffusers that I'm using for the site I'm building.
Yes 3090ti may have enough VRAM with some tweaks I think. My interest is mostly academic, so I'm hunting for tough use cases/problems to solve using this these days. So let me know if you run into problems in another domain.
Not sure what to make of this outside of it being creepily hilarious but here's a quick example where I masked out the head and ran inpainting. I'm no vision expert so I'll leave the causal factors to the ones who understand this better but I presume, with scaled fine tuning on camera trap images, this will likely get significantly better
Heres another one - I've added the mask this time for clarity
Cool, what's the species on the left (and the right)?
I presume, bobcat on the left and I prompted bobcat for the right (my initial prompt, had "bob cat" and it inpainted a domestic cat). Given the underlying nature of the training set, it's likely to have a heavy bias towards domestic creatures.
ah ok, did you do a negative prompt
These birds are really funny haha, I think the model is making some mistakes with this one for example: (Oh I can only upload 10 at a time)
Keep in mind that birds + mammals are < 20,000 species, while there are millions of insects and many animal species are nearly morphologically indistinguishable or too small to photograph reliably with camera traps. AI won't "solve" curation for species recognition without sustained investment in the research and enthusiast communities that generate the quality data it relies on. Make sure this research on synthetic data generation doesn't perpetuate the devaluation of detailed, long-term study of organisms and species in biology.
Definitely, I think that's a real risk with synthetic data approaches. Real data is always more valuable than synthetic data. The problem I'm trying to solve is how to effectively teach the model about data we already have. This can help solve partially solved problems better. For solving new problems you need new data. As Sara pointed out, the model is not universal, so I'm interested in pointing it towards more specialized or generalized knowledge.
@Abhay This looks more promising than make a video (at least qualitatively): https://imagen.research.google/video
Here is a completely random project thought. We often get developers from industry, so I think this might be a good fit. I would love, and think there would be a large market for, a 'dawn chorus app' that uses audio from the Macaulay library/xeno canto and ebird checklists to generate synthetic 10 minute snippets of bird song from a given location around the world. Fun little details like making the audio scene using generative/adversarial neural networks and the abundance distribution from checklists to make it sound real. How often the most common birds sing, etc. https://arxiv.org/pdf/2201.02490.pdf. I am preparing for a trip to Magdalena Colombia and I would love to hear sounds from the sites i'll be visiting. @Dan Morris and @gvanhorn what do you think?
I could have sworn that OpenSoundscape:
https://github.com/kitzeslab/opensoundscape
...included a bunch of stuff on realistic soundscape synthesis, albeit for the purpose of training models, not for the purpose of the soundscapes themselves. I can see no evidence of this in the README of the repo, so either (a) I hallucinated that entire conversation with Justin ~4 years ago, (b) that was true at some point, or (c) Justin's lab has a different project about soundscape synthesis.
Someone forwarded this thread to me, so I hopped in to confirm that Dan is not losing his mind, and in fact his memory is shockingly good 😄. So yes for a while we were using Brian McFee's scaper package to synthesize soundscapes for the purposes of training (and just for fun). Eventually (and currently in OpSo) we do something much simpler that's not highlighted in the docs because it's more of a practice than a functionality - when we do training we just take whatever "clean" calls we want (usually Xeno Canto, Macaulay would work fine too) and overlay them on fully annotate soundscapes that we know don't otherwise contain the species - we use these background soundscapes as negatives also. Seems to work well for our purposes.
OpSo does have the functionality/wrappers to do those overlays pretty easily along with other augmentation.
Ben's question is much more complex though and I think this is the bigger question if you really wanted a realistic soundscape, which is about the number of individuals of each species, duration of singing bouts, and temporal autocorrelation in songs/bouts - I think that's all pretty poorly understood quantitatively right now
I did an Important Scientific Experiment:
http://dmorris.net/misc/southern_roller.mp3
This is where I think the ebird part comes in, we would need to map the generative modeling to some abundance distributions. I didn't consider temporal autocorrelation in bouts. It would be interesting just as a test what parts of the natural landscape a generative model fails to reproduce. Not that I have time for any of this (nor does anyone else), but is it possible to pull in xeno canto in large batches? What's the current state of that API?
Lately I've been using a mix of xenocanto and scaper to simulate audio for training and transfer learning (also using some background soundscapes outside of the target species distribution). Scaper does allow the use of distributions to control the frequency that different call types are included.
Thus far it takes a fair amount of supervising because most xenocanto recordings are not clean.
I’m looking for good resources that outline and discuss power dynamics within research groups (particularly between the PI and research students/postdocs) - ideally I’m trying to find group code of conduct statements that outline how to address these dynamics and their potential for abuse consciously to share as an example. If you have have any related resources or or know someone who does, I’d really appreciate help, thanks!
To be honest though it is slim, I personally went through these conversations with my PhD Advisor and we created an expectations document that immediately was not followed through upon
we tried to utilize published literature like this: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006914
here is our code of conduct we review with students when they enter the lab. https://wiki.weecology.org/docs/lab-policy-procedures/code-of-conduct/
Hi Adam! Similar to other responses, our lab also has a code of conduct / shared values that we go over with incoming members. Happy to share it with you if you PM me. Also passing on a message from Anne Simonis, who wrote our document, on some resources that that served as inspiration:
"In 2018, the National Academies produced a report on sexual harassment in academia, which included a tangible list of recommendations described here: Sexual Harassment in Academic Sciences, Engineering, and Medicine - YouTube
Dr. Kate Clancy clearly explained why defining our shared values & code of conduct is needed when she was testifying to Congress:
"The way we've tried to fix this problem isn't working, we have decades of evidence to prove it. Let's move away from a culture of compliance and towards a culture of change.......I hope you will join me in encouraging universities and other science workplaces to take a values-into-action approach to eliminating sexual harassment. That means locational, contextual solutions that create respectful and equitable climates for everyone. That means focusing on the behaviors we want to see, not creating fear around the legally actionable ones. And that means creating confidential avenues for women to speak, and to be heard."
I like how Dr. Clancy's lab Code of Conduct has defined shared values, best practices, links to resources, etc.
Here's another short, sweet example from Penny Chrisholm's lab."
@Devis Tuia (or anyone) do you have a pdf of the book. I've got full access through my University, but after authenticating, Wiley's link just leads to 404 corrupt.
I don’t have the final version, but I have some kind of intermediary one.. I can send that one your way or inquiry for a more official pdf… I let you know, Ben
hope you will like it! It was quite some work.. unfortunately, no chapter on conservation, even though we talk about our Namibian domain adaptation experiments in the domain adaptation chapter
Hi everyone, I'm wondering if anyone on here has an experience with user research or user testing for conservation technology 'products'? Please get in touch, I'd love to discuss methods!
we've worked with some AMAZING service designers who led lots of user research for us
Hi Steph! Anyone you’d recommend chatting to?
yeah, but let me just check they're happy to talk!
This also might be a discussion to start on wildlabs too - you're likely to get more of a cross section of the 'product' developer part of our conservation tech ecosystem there.
HI Steph, that's great - I'll post on wildlabs too, I look forward to hearing from you!
Hi Sicily! Yes I have someone in mind that I'll dm you directly about!
Upcoming AI for Environmental Science Meetings, please retweet to get the word out: https://twitter.com/scotthosking/status/1591000813845245952
https://www.youtube.com/watch?v=3rQi2uNqwxk
Though call 🙂 On one hand I believe humans are equal to any other living being and we don't have a special status (that we currently assign to ourselves).
On the other hand, I believe killing other living beings for your own gain is natural. This happens in all animals and plants, for food, territory, etc.
The main problems these days IMO are, the scale at which we are killing, sometimes even seemingly pointlessly. And the fact that many people think they are better then other species, better than nature, or even not thinking they are part of nature at all. Which is total BS in my eyes :)
It is a good starting point for discussion. As Thijs, I don't think we are special and I don't think we can assign ourselves a higher status compared to nature/animals, but in the end for medicine we are special.
I don't think that we can compare "killing for surviving" which is normal in nature with "killing for surviving" as humans are applying. We kill lots of beings for everything from the potential mortal virus to lipstick.
I cannot say "save the monkey and doesn't matter if you cannot find the cure for cancer" because I want to have the possibility to survive in case of cancer, but I think that the problem is that we give for granted that we can kill everything just for our wellbeing.
In any case, it is a really complicated topic, I don't want to kill animals for my well-being but I want to survive in case of disease.
20 minutes ago, I took the medicine for the headache that probably has been experimented on animals, I am feeling bad for the animals and I am feeling relieved from the headache.
For anyone attending AWS re:Invent this week, I'm presenting a Dev Chat at the Developer Lounge (Booth #2818) at 2pm on Tuesday about developing a web application using AWS Amplify Studio, Amazon Rekognition and Amazon SageMaker to engage Explore.org viewers for human-in-the-loop machine learning.
Is anyone coming to NeurIPS? I'm applying to PhDs this cycle, would love to chat about potentially working together in AI+conservation policy!
I'll be at NeurIPS and am looking for PhD students! I will only be here tomorrow, so please feel free to DM me and maybe we can meet up
Dear all, a call for a 3-4 year chair in our University around AI. Could obviously be developped for conservation.
Call for Research Fellow Chairs 2023
MIAI, the Grenoble Interdisciplinary Institute in Artificial Intelligence (https://miai.univ-grenoble-alpes.fr/), is opening three research fellow chairs in AI reserved to persons who have spent most of their research career outside France (see below). MIAI is one of the four AI institutes created by the French government and is dedicated to AI for the human beings and the environment. Research activities in MIAI aim to cover all aspects of AI and applications of AI with a current focus on embedded and hardware architectures for AI, learning and reasoning, perception and interaction, AI & society, AI for health, AI for environment & energy, and AI for industry 4.0.
These research fellow chairs aim to to address important and ambitious research problems in AI-related fields and will partly pave the way for the future research to be conducted in MIAI. Successful candidates will be appointed by MIAI and will be allocated, for the whole duration of the chair, a budget of 250k€ covering PhD and/or postdoc salaries, internships, travels, … They will be part of MIAI and the French network of AI institutes (comprising, in addition to MIAI, the AI institutes in Paris, Toulouse and Nice) which provide a very dynamic environment for conducting research in AI.
Eligibility To be eligible, candidates must hold a PhD from a non-French university obtained after January 2014 for male applicants and after 2014-n, where n is the number of children, for female applicants. They must also have spent more than two thirds of their research career since the beginning of their PhD outside France. Lastly, they should be pursuing internationally recognized research in AI-related fields (including applications of AI to any research field).
To apply Interested candidates should first contact Eric Gaussier (eric.gaussier@univ-grenoble-alpes.fr) to discuss salary and application modalities. It is important to note that candidates should identify a local collaborator working in one of the Grenoble academic research labs with whom they will interact. If selected, they will join the research team of this collaborator. They should then send their application to Manel Boumegoura (manel.boumegoura@univ-grenoble-alpes.fr) and Eric Gaussier (eric.gaussier@univ-grenoble-alpes.fr) by March 11, 2023. Each application should comprise a 2-page CV, a complete list of publications, 2 reference letters, a letter from the local collaborator indicating the relevance and importance of the proposed project, and a 4-page description of the research project which can target any topic of AI or applications of AI. It is important to emphasize, in the description, the ambition, the originality and the potential impact of the research to be conducted, as well as the collaborations the candidate has or will develop with Grenoble researchers in order to achieve her or his research goals.
Starting date and duration Each chair is intended for 3 to 4 years, starting no later than September 2023.
Location The work will take place in Grenoble, in the research lab of the identified collaborator.
For any question, please contact Eric Gaussier (eric.gaussier@univ-grenoble-alpes.fr) or Manel Boumegoura (manel.boumegoura@univ-grenoble-alpes.fr).
**************
Hey all, I run a reading group/talk series for robotics (and robotics-adjacent) topics applied to sustainability. We have a slack if you'd like to join yet another one.
We're having our first talk of the year from Marius Wiggert, a PhD student at UC Berkeley and member of the Climate Change AI core team. It's at 1PM EST on 1/24/23 on zoom and you can add it to your gcal here. The talk description is below.
There are hundreds of solutions to protect our planet, but many of these are too expensive and labor intensive to be applied at the necessary scale. Building on lessons from manufacturing — where robotic automation has had wide success in scaling solutions — many have started to explore the efficacy of robotic automation for climate change solutions. These applications range from scaling reforestation with drones to building solar farms with robots. This webinar will discuss specific applications of a machine learning (ML) in the field of robotics for climate applications in conservation, restoration, and climate change mitigation. In particular, we will discuss the application of ML to coral reef monitoring and restoration and ML to enable autonomy within floating seaweed farms.
Hey AI folks! Does anyone have a cool image of an ai for conservation application you wouldn't mind us using for a poster/event? i.e. animal with bounding box is usually synonymous with ai in conservation. We are about to launch our Ai for Conservation office hours sessions again and would love a great image for the poster/landing page. We'll include full credits and we'll link to your work if you're kind enough to let us borrow your image!
I mean, do feel free to also use this as shameless opportunity to share your favourite photos from your projects...
Two of my favorite pictures, these are from the project we did in Gabon to use our AI smart cameras to test if we can use this tech to mitigate human-elephant conflicts.
In the end these elephants that our smart cameras detected (a mother and two young) broke into locals plantations and destroyed many crops.
So this picture is basically the moment when these elephants started an all-you-can-eat buffet for many weeks 🙂
Of course I can also draw the bounding boxes on them if needed
Hi, Here are some pictures from our WildCap project at Uni Stuttgart. There are P. Horses and G. Zebras pictures, all from drones, either -- some from our own autonomous ones. I will be glad if you could credit them with the project link: https://www.aamirahmad.de/projects/wildcap/
AMAZING - thanks so much Thijs, Aamir and Lukas (who DMed me). These are better than i was even hoping! I now have wildly high expectations of you all jumping into our #tech4wildlife photo challenge later in the year...
forgot to mention, the filenames of the images have a short description of the location, etc.
…. can audio/spectrograms count??
Camera-trapping gets all the hype😂 (not biased at all….)
i'm offended you even had to ask. We are equal opportunists 😂
the original post has “animal with bounding box”! 😂
I shall go through my files and see what I can dig up
fair fair. it was intended as an inspiration prompt rather than being prescriptive
You have used BearID photos before (thank you!), so I won't submit anything this time! 🙂
A gentle reminder to machine learning students in the community. I just got absolutely burned by this for like the 3rd time in a week. Different behavior in model.eval(). Unfortunately, the model works about 20% with batchnorm. I want to recommend this set of rants on the dangers of batchnorm, https://www.alexirpan.com/2017/04/26/perils-batch-norm.html https://towardsdatascience.com/curse-of-batch-normalization-8e6dd20bc304, https://twitter.com/jacobmbuckman/status/1391284966340898816.
Twitter
I've been fighting with catastrophic performance due to batch norm for many years now, going through haphazard workarounds like leaving the batch norm layers in training mode during inference (ugh) or replacing them with instance norm (big compromise). It basically failed on me for whichever dataset I fine-tuned the model on that was not ImageNet (i.e., every single experiment).
https://www.acm.org/diversity-inclusion/words-matter
Hello 🌊!
First of all I love this place. Thanks for making it, being here and sharing! I am learning a lot!
My question:
Any interesting jobs/projects that you know about?
I am currently in the opportunity searching/job application process, and things are moving along well. Some sweet companies and interesting opportunities are looking likely.
But is not in the conservation/biodiversity.
So, I am wondering if there is any interesting job or opportunity floating around, before I sign off my time to another field. I would really love to find something more inline with what this community is about. 💙🌍💚
I am looking for Software Engineering roles, and making my way into machine learning. I have a pretty varied background and a fair bit of experience, so can usually contribute in many ways, but would love it if the bulk of the work was technical!
Part time / short term projects work too!
My email: kalindi.fonda@gmail.com, here is my resume, and here is my calendar availability, if anyone wants to set a time with me. Feel free to share.
Thanks a lot!
🌱
👋 @Kalindi Fonda! You may want to check out the https://climateaction.tech/ community as well. They're a community that only revolves around jobs in the climate space, and majority of the roles are in tech. 🙂
Thanks, joined this already, very lively and useful. Similarly https://workonclimate.org/ is a good community too (I think that's the one @Dan Morris mentioned during the call right?)
terra.do is another one - https://terra.do/climate-jobs/job-board/
Yes, I just discovered the Work on Climate (WoCl) community and went to one of their monthly meetups in Seattle. I would say the Slack chatter is less technical than this one, so I don't hang out there the way I hang out here, but it's really nice that there's critical mass to have local meetups. I met a bunch of interesting people!
Hey everyone, we just published an article (a follow-up of a quick POC we created last year) on using ML to segment copepod prosomes and lipid sacs (or looking at how fat zooplankton are) 🦐🔬🌊
Basically, we're using AI to analyze high-res microscopy images and help researchers better understand how these little critters are affected by climate change and other environmental factors. It's pretty cool stuff.
Our model is making the analysis process faster and more accurate, which means we can gather insights and make progress on marine biology research even faster. 🌍🐟
A paper utilizing the results is on its way (and I’ll be sure to share it when it arrives, but it will take some time)!
Please let me know what you think in the comments! 👍 If you have any questions I’ll be happy to pass them on to our ML team, feel free to comment below or DM me 🥳
Lastly I'll be grateful for recommending whom I could reach out to share our work in this area (if they are not here). Building our Data4Good program we constantly seek partners to help them accelerate research and solve sustainability challenges.
Many thanks! 🤝
I cannot judge the novelty in the CS fields but for someone who works on millions images of plankton, this is the way forward. We are storing video and light images, CT scans and other visuals and do not have the people power to draw all the information out of it. This is a great example of transfer of learning and knowledge.
@Daniela Schmidt - it's great to read this, maybe this can be useful for you and your research? Or perhaps you have another project in mind? If you have a similar dataset or are interested in how our AI & Research team can assist in your project, let’s talk.
This is not AI related but just a fun activity to do this time of the year 😂 - https://twitter.com/MMMletsgo/status/1628001241845612551?s=20
Twitter
Any volunteers to help us submit an AI for Conservation happy hour as a social event at ESA?
Our lab will be there and we could help out if it's useful
I was wondering if anyone had any reading recommendations (blog, paper, book, etc..) with best practices or clear thinking about building statistical models on top of perception model outputs?
For example, should we be augmenting the stated assumptions of common ecological models (i.e., occupancy) when sensors and perception models are used? What would those stated assumptions look like? How would they change as fine-scale confidence scores are aggregated into detection histories (at a temporal scale that is longer than the sensor processing scale)? Also, how do thresholding decisions affects those assumptions?
And to extend the thinking in Rhinehart et al. (2022) how should I be thinking about perception uncertainty and error propagation when confidence scores are aggregated into detections?
Great questions, and ones our group thinks about a lot. I don't really have much to recommend, as I think (at least in ecology) these questions are right on the edge of new research ideas. An important question right away is whether you try to interpret the output of classifiers as a (calibrated) probability or not - the techniques that would come next bifurcate right away (in my opinion) depending on the answer to that question. One way to approach this whole thing is to jump off of the history of occupancy/N-mix/etc modeling, in particular work that allows false positives - I usually recommend the two AHM books (Kery and Royle) for a comprehensive review of what's been done in that space.
Thanks! Those are really interesting suggestions. I have spent a little bit of time using false positives rates to adjust aggregated detection frequencies. But I have been using coarse false positive rate estimates (i.e. no spatiotemporal variation in estimates of fpr) so the result is just a rescaled dataset.
I'm excited to see papers from the next few years. It will be interesting to see how some of these questions are resolved (or at least moved forward).
Confidence scores can be deceiving, especially in classifier networks, due to the softmax function being applied. The confidence is the pseudoprobability of the highest scoring class and gives an approximation of the aleatoric uncertainty with respect to the training data distribution, but does not give a good indication of the epistemic uncertainty (aka does the network know what it is saying or is it guessing) -- there's a blog post explaining that really well: https://towardsdatascience.com/my-deep-learning-model-says-sorry-i-dont-know-the-answer-that-s-absolutely-ok-50ffa562cb0b
If a perception model is applied on data far outside of it's training data distribution, one cannot trust the confidence score at all. There will typically be large number of highly confident false positives as well as false negatives. In the blog they use dropout to create a random ensemble of networks and measure the variance across all outputs on the same data to measure epistemic uncertainty. But that's typically hundreds of times slower than running a single network and only suitable for offline methods/processing.
In our own work we did some research on treating a perception model as a "sensor" - in this case we need to model the localization error the "sensor" makes. https://doi.org/10.1109/LRA.2018.2850224. i.e. if a true positive has been detected above a certain confidence threshold, how accurate is the localization in pixel space. That can be well approximated by a Gaussian distribution, but needs to be measured across a sufficiently large evaluation set.
I appreciate the replies here. Both have helped me move my thinking a bit on this topic.
I wanted to toss this out there. If anyone would like to form a reading group focused around these topics please send me a message. I'm thinking a paper every 2 weeks or so? We will probably have to do a bit of digging to find relevant work (false positive estimation, uncertainty quantification, model calibration), but I think it will be nice to gather my thoughts on the topic.
Hi 👋 just wondering if anyone here could help with studying Data structures and Algorithms? I have an interview coming up, and am unsure where to begin. Thanks
You might take a look a this https://runestone.academy/ns/books/published/pythonds/index.html
Can we add the output of your google street view tree ID model into this database? 😀
🙂 One of my first web+data+community projects was called Urban Edibles and we eventually rolled our data into Falling Fruit. And FF actually still uses the Code of Conduct we wrote for foraging! This was before smart phones...
https://docs.google.com/document/d/1SupIGQKC5Vgi3VYkdIQc05y_S7jSoZ4RTS-CzwlEAMY/edit
Also the city of Portland re-inventoried all of their street trees (by hand) over last year. I'm not sure when the survey will be available, but should be some nice fresh labeled data for your model!
Lasers, drones and AI: the future of weeding https://www.bbc.co.uk/news/business-64742513
Has anyone played around with using https://codecarbon.io/ in their code base? looks cute.
Happy Friday everyone! Here’s GPT-4’s advice on how you can make a difference in the field of conservation science using the power of AI. 🤖 🐿️ 🌮
Total fiction. You can't find guinea pig-sized lab coats anywhere. I have tried. We settled for hand sewing surgical gowns, and the little lab assistants seem just as happy with those. 🙂
From GPT-4:
“Once upon a time, in the futuristic kingdom of Conservationia, there was a mad (but well-meaning) scientist named Dr. Greenie McSavetheplanet. Dr. Greenie was on a quest to save the world from environmental doom by using the most powerful weapon known to humankind - Artificial Intelligence!
In his secret laboratory, surrounded by lasers and guinea pigs wearing tiny lab coats, Dr. Greenie devised a wacky plan to harness the power of AI to revolutionize conservation science.
Step 1: Dr. Greenie trained a legion of AI-powered robotic squirrels to identify and monitor endangered species in the wild. These fluffy cyber-critters used advanced fur-recognition technology to track the most elusive of animals, even the rare and mysterious Spotted Gigglephant.
Step 2: He created an AI called GreenGenius9000, which could analyze the tweets and burps of every living creature on Earth to detect patterns of habitat destruction. GreenGenius9000 could also predict when and where conservation efforts would be most effective, using a top-secret algorithm that somehow involved tacos and interpretive dance.
Step 3: Dr. Greenie made use of AI-powered drones shaped like oversized bumblebees to replant entire forests at lightning speed. These "Bumble-Boomers" could buzz around the world, planting seeds and high-fiving birds with their tiny drone wings, all in the name of conservation.
With the power of AI and a sprinkle of silliness, Dr. Greenie McSavetheplanet transformed the field of conservation science, making the world a better, greener place for all living beings.
And they all lived happily ever after, except for the occasional AI squirrel going nuts - but that's a tale for another day!”
Very cool, and ...
a legion of AI-powered robotic squirrels to identify and monitor
borderline eery 😂
I wonder whether this webinar would be useful to anybody here. I think it's free:
The International Environmetrics Society (TIES) has launched the TIES Webinar Series on Data Science for Environmental Sciences (DSES).
Our next webinar will be on March 31, at 9 am Central Time...
You can virtually access the webinar and register via our website: www.environmetrics.xyz
Speaker: Claire Monteleoni, Choose France Chair in AI & Research Director (INRIA Paris) and Associate Professor (University of Colorado Boulder)
Title: Machine Learning for Climate Change and Environmental Sustainability
Abstract: Despite the scientific consensus on climate change, drastic uncertainties remain. Crucial questions about regional climate trends, changes in extreme events, such as heat waves and mega-storms, and understanding how climate varied in the distant past, must be answered in order to improve predictions, assess impacts and vulnerability, and inform mitigation and sustainable adaptation strategies. Machine learning can help answer such questions and shed light on climate change. I will give an overview of our climate informatics research, focusing on challenges in learning from spatiotemporal data, along with semi- and unsupervised deep learning approaches to studying rare and extreme events, and precipitation and temperature downscaling.
Bio: Claire Monteleoni is Choose France Chair in AI and Directrice de Recherche at INRIA Paris, an Associate Professor in the Department of Computer Science at the University of Colorado Boulder, and the founding Editor in Chief of Environmental Data Science, a Cambridge University Press journal, launched in December 2020. She joined INRIA in 2023 and has previously held positions at University of Paris-Saclay, CNRS, George Washington University, and Columbia University. She completed her PhD and Masters in Computer Science at MIT and was a postdoc at UC San Diego. She holds a Bachelor's in Earth and Planetary Sciences from Harvard. Her research on machine learning for the study of climate change helped launch the interdisciplinary field of Climate Informatics. She co-founded the International Conference on Climate Informatics, which turns 12 years old in 2023, and has attracted climate scientists and data scientists from over 20 countries and 30 U.S. states. She gave an invited tutorial: Climate Change: Challenges for Machine Learning, at NeurIPS 2014. She currently serves on the NSF Advisory Committee for Environmental Research and Education.
📣 Hi everyone, the Appsilon team has been working on improving and adding new features to our Mbaza AI, and life's good.
However, we've recently received reports from some of our users about a "JavaScript error occurred in the main process" message.
We've tried to reproduce this error in-house, but we haven't been able to do so 😮💨. Therefore, we're reaching out to you to ask for your help. We'd appreciate it if you could download and install Mbaza AI and see if you encounter the same error.
If you do get the error, please let us know the following information (either DM me here or email at andrzej@appsilon.com please):
- Your hardware specifications (CPU, RAM, etc.)
- Your operating system (including the version number)
- Any other relevant information you think might help us diagnose the problem
We're committed to resolving this issue as soon as possible, and your help would be greatly appreciated. Thank you in advance for your time and effort.
🔗 Download link in our GitHub.
@Andrzej Białaś - does the detection happen on device or by calling a cloud hosted API endpoint
read through the docs, its offline. Its working fine at my end. Likely the error could be because of incompatible JS versions. or incorrectly installed files. Maybe you could ask your technical team to take a remote access, and re-install the required packages.
Thanks you Aakash, I'll pass on that feedback. I'm sure we'll get to the bottom of this.
👋 Hi everyone, me again, I'm looking to crowdsource some inspiration to make Mbaza better, and I'd like to look through what's available on the market.
❓ Can you please share existing AI solutions that contribute towards biodiversity conservation and monitoring, or otherwise help solve us global sustainability challenges in the 🧵?
At this stage anything goes, I'm doing a high-level brainstorming, I'll be grateful for any suggestions.
Machine learning has fairly recently (in the last few years) made its way into the field of underwater acoustics. This has made things like source range estimation/localization (among other tasks which are useful for biodiversity monitoring) much easier and great candidates for automation. It would be great to see these techniques integrated into a functional system!
Check out the Conservation Tech Directory 🙂
For camera traps in particular, here's a list of all the AI-powered systems I know about (including Mbaza!):
https://agentmorris.github.io/camera-trap-ml-survey/
Though it's not totally AI-centric, @Tessa Rhinehart has curated a great list of bioacoustics software, much of which includes some AI functionality, though in the acoustics space, you get into a much fuzzier area around what "AI" is (is spectral template matching "AI"?)
https://github.com/rhine3/bioacoustics-software
All that said, the largest cluster of AI-powered tools being used for conservation is almost definitely in the geospatial space (everything from platform-sized things like Earth Engine to desktop tools like eCognition), and I don't know of a comprehensive survey. Nudge, geospatial ML community?
Taking a cue from Carly and Dan, typical use-case in earth observation includes:
- Change detection from space - identification of illegal logging incidents
- Wildfire detections using IoT sensors - an example will be the platform provided by Dryad.net
- Identifying poaching incidents through acoustics - gun-shots through a network of sensors
- Solving the last-mile problem of real-time collection of camera trap images. This is an intensive manual task performed by beat officers. Setting up mobile towers in ecologically sensitive areas can be mis-used by poachers. So typically you can look at LoRaWN tech or Mesh networks for hopping signals and edge computing. Though the bandwidth is limited.
- Habitat management (grasslands) and watershed management during dry seasons to ensure that prey species have adequate nutrition.
Another use cases are getting an accurate count of ungulate population from thermal drone cameras.
With DeCaTron we have built a data fusion layer on top of the information collected from camera trap images. So you can incorporate GIS information as well as other data from line surveys and daily monitoring along with EO data to understand the spatio-temporal distribution of the population. As well as take proper corrective measures in terms of patrolling of sensitive areas of the reserve forest and habitat management.
Thank you very much everyone, it's been very helpful in my search 🙏 💯
Edit: I realized I forgot to reply... 🫣
Hi!
We just released a new Python SDK to convert existing models (ONNX, PyTorch and Tensorflow) to C++ libraries optimized for edge deployment.
https://www.edgeimpulse.com/blog/unveiling-the-new-edge-impulse-python-sdk
I'm working on a couple of integrations with SageMaker, Hugging Face, Nvidia TAO Toolkit, etc...
Let me know if you have a cool use case using these tools so I can test the integration.
I tried to use the SDK, but I got some authentication error 🤔
Is this feature something that needs to be enabled in my account?
Hello @Thijs!
No it should be pretty straight forward. Have you properly set your EI API Key?
Feel free to ping me if you have any issues
Suggestions for podcasts?
So, during out social two months ago, many people mentioned their love of running (aided by audio/visual support).
I decided I'll give it another go, and pair it with some good listening.
So what are some interesting (and maybe relevant to this group) podcasts and audio books to take on a run with me?
🏃♀️ 🎵 🌱
Thank you
I'm all fiction in my podcasts and running (and leveraging that this is the random channel):
- Spark Hunter (AI spy thriller)
- 36 Questions (musical/love story)
- Wolf 359 (multi-season space opera)
- Limetown (creepy, well produced)
- How to Win Friends and Disappear People (a vampire and a serial killer become neighbors)
- Fathom (undersea disaster sci-fi)
- Levar Burton Reads (because adults need Reading Rainbow too)
We've got a list going here!
https://wildlabs.net/discussion/what-conservation-and-tech-podcasts-are-you-listening
Big Fan of 'Machine Learning Guide' when wanting to brush up on technical skills, has a great host
I confess i've been somewhat ambivalent about all of the GPT innovations, most just have the veneer of productivity or insight. I thought this one was a bit more interesting. The scispace co-pilot (https://typeset.io/copilot-chrome-plugin). Helpful for scanning papers quickly. Here is an example, that happens to include @Alexandre Lacoste on pretraining remote sensing data. Quick q&a on pdf in browser.
I was currently building myself a tool for digesting the literature using GPT4. Let’s how this one is doing.
The tool that I’m trying to build is slightly different though.
Given a new idea described as text, it searches the web for all papers that might be related. It quickly ranks them by relevance by reading the abstract using GPT 3.5. For the top 10 papers, it reads them in depth with GPT-4 to find the key differences and similarities it also summarizes the papers and important details. Then you can prompt gpt4 with the summary of the top 10 papers and have an in-depth discussion about recommendations for next steps.
@Alexandre Lacoste that sounds like a really great approach. Do you plan on open sourcing what you’re working on? It’s exactly one of the issues our science team is dealing with but they don’t have the coding prowess to build their own solution like yourself. Good luck and thanks for your contributions in this space!
If I can find time to finish it I’ll definitely open source it
That’s awesome! Good luck! Did you start with a framework like LangChain or are you building it by hand similar to AutoGPT?
I started building it by generating code from prompting GPT-4, asking it to build that tool 😉
I recently learned about langchain and I’m starting to use it.
Oh that’s awesome! Good luck on the project. Let me know if you make more progress! I like your approach so far 😄
👋 Hej Folks, I'm super excited to share two pieces of news, both Mbaza related.
- We just hired a dedicated product manager to focus on Mbaza's development. I'll introduce him to this wonderful community after he's onboarded to the team, but we are all very excited for the coming months. 🎉
- Mbaza has been placed on the 2022 IRCAI Global Top 100 international list of AI solutions for the SDGs! 🎉
Hi all 👋 My name is Chris and I discovered your slack server after watching the Caltech webinar a few weeks ago on AI / ML / Machine Vision. I’m the head of IT for a company called Terraformation and we’re trying to fight the climate crisis by creating an Accelerator for Reforestation projects. We work with partners around the world in an effort to grow more biodiverse native forests. Anyhow, I just wanted to say hi and introduce myself as I’ve been lurking and checking out the projects others are sharing. Like many of you, we’re excited about these new language models, machine vision products and we’re excitedly trying to find ways to integrate them into our workflows. Enjoy your day everyone! 🌺
Hey ya'll we just dropped a new publication in E&E, please give it a look-
"Application of a deep learning image classifier for identification of Amazonian fishes"
https://onlinelibrary.wiley.com/doi/full/10.1002/ece3.9987
Props for sharing the data! I downloaded, super-straightforward, posted a link here:
https://lila.science/otherdatasets#images-marine-fish
I feel like 91% of my posts to this Slack now are "I downloaded your data set, tried it out, and added it to one list or another".
Hello, my name is Dan, and I'm a data-hoarder.
hi all.. was wondering if anyone is using the twitter academic api to download data for their projects.. with all the upheaval going on on twitter not really sure if anything has changed there.. is the monthly limit still at 10mil ? or is it capped lower like the regular api? thanks if anyone has clues
Hi everyone 👋 This afternoon I have an interesting clip for you all. 👀 A few months ago I posed challenge to two student teams at Oxford University to see if Megadetector V5 could be configured to run inference on live streaming video frames from an RTSP / RTMP source (i.e any live watering hole camera from a publicly accessible YouTube live stream). The objective was to ingress a live stream, process each frame on a local laptop (+ a little GPU support) to output the bounding box as an alpha layer, and then output that inference live to a new stream that could then be re-streamed back out to YouTube from the same laptop - essentially allowing anyone with a YouTube link and a laptop to process the feed to count the number of animals visiting a particular waterhole, assess absence / presence in draught situations etc (+ lots more things).
I wasn't too sure if it would be a laggy experience, but they pulled it off. Here's a little demo or a live restream running at 24fps with real-time detection. @Sara Beery - the answer was yes
Really cool!! Pardon my ignorance, can you please explain why/what you were expecting to be laggy?
It's to do with the speed of the frame capture and processing. It has to process each frame (1920x1080px) at 24 frames per second, generate the bounding box, draw it into a new video frame and then stream it back out to YouTube all in real time, all on the same laptop. I was expecting maybe 15 fps, or even <10, but it's possible to get a smooth experience and just point it at live URLs for instant results
Nice work! Check out this demo I saw recently… applied to these camera traps it could be pretty neat. https://twitter.com/mckaywrigley/status/1651291367224807424 The author sometimes open sources his work so I’ve been following their github as well: https://github.com/mckaywrigley
Out of curiosity, what laptop/GPU configuration did they manage to pull it off on?
i5 with an RTX 3070 GPU (laptop). Pretty decent but certainly not desktop grade
Very cool! I have done something similar with other object detection models, but not Megadetector (I also didn't re-stream it). This would be a great frontend to the Bearcam Companion (overview) I am working on for the Brooks Falls Bear Cam!
Are you running the model at the standard/training resolution (1280px on a side)? That's the default if you use our inference code. If so, I'm quite (pleasantly) surprised by 24 fps! I typically get around 12-13 images per second on an RTX 3090, sometimes (rarely) as high as 20, and if you're getting 24 fps on an RTX 3070, that suggests I'm more limited by disk I/O than I thought I was.
The pessimist in me is concerned about the instability of some of the zebra detections, not sure if that's just a confidence threshold issue, or if that indicates that the model is getting run at a lower-than-typical resolution. What confidence threshold are you using?
This looks great. We did something similar for outreach demos at museums where we'd have a thermal camera rigged up and a live YOLO classifier running on the thermal + RGB streams. It worked fine on a laptop 1060 or so. Generally didn't need to push the resolution too much. This is also how we worked with drone imagery - fly the drone with a WiFi antenna and have a good receiver on the ground. Drone sends back an RTSP stream over e.g. ROS and then we pick it up on the ground and ML happens (or in the air if you prefer, to send back annotated images). Never tried to re-stream it over youtube but I guess you could also easily do it with with something like OBS Studio on Linux (I've used that for streaming games to friends in the past).
Hi everyone. In case you have missed our LinkedIn post, here it goes. TL;DR we have opened up our recent work on learning extremely accurate detectors for animals using purely synthetic data! Have a look here:
https://www.linkedin.com/posts/ahmadaamir_syntheticdata-robotics-drones-activity-7061981286113927168-COKd?utm_source=share&utm_medium=member_desktop
Aamir this is awesome. Re GRADE - would you classify this as 'generative AI' given that you are effectively asking it to render life like zebras as your training data?
Thanks, George! To answer your question -- No, it is not 'generative AI'. The synthetic data generated by GRADE (in our current public version) is not hypothesized (i.e., not sampled from a distribution -- which in turn could have been learned somewhere). What we are doing is purely classic here. We need to start with 'rich' 3D models of animals, with texture, objects or humans. Synthetic images are then generated using ray tracing from any queried view-point in GRADE. Other sensor data (typical for most robots like IMU, odometer, etc) is generated from the known sensor models of various sensors. GRADE is mainly focused on robot perception tasks, therefore I believe using Generative AI-based methods (hypothesizing synthetic data) to generate data for training networks for other downstream tasks in this case could be tricky -- perhaps even dangerous for safety-critical application like collision avoidance. Therefore, we currently have classic approach for it where we use the physics/rendering engine to generate data and add noise on top of it using known sensor models. Perhaps @Elia Bonetto could add if I missed something.
I am running a 3-day camp for high school students introducing them to ML/AI/NLP. These students have some prior experience with programming (they know what variables are, they know how to write if else statements and loops).
Does anyone have any advice on topics I could include?
Any ideas for hands-on coding activities?
(PS: I asked chatGPT but the result wasn't great!)
I've had success with this activity in the past! I thought it was a great explanation of model training, training data, and the ways that poor data can impact model results. And the kids I did it with seemed to have fun! https://code.org/oceans
For NLP / large language models https://gandalf.lakera.ai/ check out this website. It’s a pretty fun game where you have to interact with a LLM to make it reveal a password. Great for intuitions about safety and or privacy of these models.
I would recommend checking out some of the Kaggle datasets!
https://www.climate-forests.org/comment
Does anyone have experience uploading (large) datasets to google earth engine? I was asked to upload our shapefiles of tree predictions (thousands of shapefiles, tens millions of polygons), and haven't found much luck in using their python api https://stackoverflow.com/questions/48148800/possible-to-upload-shapefiles-in-google-earth-engine-python-api, looks like I should actually create a google bucket and upload as an 'asset' (https://developers.google.com/earth-engine/guides/command_line#upload)?
Uploading as an asset to cloud storage, is the best practice as the docs recommends,
Although, if you chose the Google Earth Engine for querying on map only, as a "free" substitute for GIS, I would suggest to try geopandas.
if you already familiar with padnas package and Python programming language, the learning curve will be a walk in the park.
The obvious disadvantage is lack of cloud features the Google Earth Engine have.
I have uploaded large(ish) files, single digit GBs, as assets through their GUI (code editor), but if you have anything larger this might not be the best solution.
I had successful experiences uploading large amount of raster data to GEE via Google Cloud Storage (GCS) buckets. Not sure if the same procedure applies to vector data.
- Upload the data to GCS bucket e.g. with rclone.
- Create manifest .json file with the uris to the files in the GCS bucket.
- Submit an ingestion job with the manifest.json file using earthengine upload
oh... I forgot, on linux system, you can use the fuse to mount the bucket as nfs, and access it as a system folder.
https://cloud.google.com/storage/docs/gcsfuse-quickstart-mount-bucket
perfect, i'll post any questions, was just confirming this is the right workflow thanks @Nico Lang
Thanks everyone. Up and running on earth engine. https://code.earthengine.google.com/be3907309cf76e23b2ad8d132db41072 . Recommendations for web apps using google app engine/heroku versus using geemap and viola to host jupyter notebooks? Just the best way to show users our data overlayed on google earth assets.
recently I got a gentle hint from google to start my new applications on cloud run or GKE since they plan to retire the app engine (it is an outsourcing service wrapped in google UI), the migration documentation should be on air already in their blog.
https://cloud.google.com/appengine/docs/flexible/run-flex-app-on-kubernetes?tab=python
https://cloud.google.com/appengine/docs/standard/cloud-run-for-gae-customers
@Ben Weinstein how can I access the dataset?
sure, but which part are you looking for. Still uploading and make sense of how best to put the images together. We will be releasing about 25 sites worth of tree crowns for NEON. Just testing now with one site from Harvard Forest. NEON is still uploading sensor data. Google earth object is public https://code.earthengine.google.com/f33018bbc63baf4e2d7e3613d7eb0edf. But you should know that this is still pre-release.
Cool thanks! I will play around with it!
Hi everyone!
We (@Philipp Spitzer and I) are researchers at Carnegie Mellon University and Karlsruhe Institute of Technology (KIT) seeking to better understand how explanations for artificial intelligence (AI) make predictions more useful, usable, and trustworthy to end-users. To investigate this, we are running a task that asks users to identify images of birds, and then collaborate with AI and generated explanations to arrive at a final identification. Participants will be shown images of birds commonly seen throughout North America and view information from AI such as natural language explanations or example-based explanations to rationalize the bird species classification. The data collected in this research study will be used to identify important characteristics of explanations for image classification under uncertainty when the decision-makers have varying levels of expertise and help us design better explanations for artificial intelligence in general.
This task is part of a research study by Katelyn Morrison at Carnegie Mellon University, IRB STUDY2022_00000062. Participation is voluntary. Participants will be asked to complete a 20-minute-long web-based task. After completing the entire task, participants will be compensated an Amazon gift card of 5 USD at a rate of approximately 15 USD/hour.
If you are interested in participating in this study, please view the study here (https://explanations-9a4b7.web.app/). First, you will be presented with a screening questionnaire to determine if you are eligible to participate in this study. Along with the criteria listed below, you must pass the domain knowledge test (be familiar with birding and/or do bird-related work) and classify 2 of the 3 common ”easy” bird species shown to you correctly. If you are eligible to participate in this study, you will automatically be shown a consent form page to agree to before doing the study task.
Eligibility
- 18 years+
- Fluent in English
- Living in the U.S.
- Previous experience with birding (participated in birding activities and/or doing bird-related work)
- Browser Requirements: Chrome/Safari/ Firefox
Thanks a lot in advance. If you have any comments, questions, or concerns related to this study, please reach out to me (kcmorris@cs.cmu.edu).
I'll give it a go, and pass it along to a few birders. Always happy to help fellow Tartans!
Done. However, these are not the birds I am familiar with! 😂
I did it. Have thoughts if interested, don't want to dilute the thread.
Thank you to all who have participated in our study so far! ✨ We have now opened up the study to those residing in Canada and Europe!
Do you guys use Threads? I’m here 😀 I'm on Threads as @arvinsunboy. Follow my threads. https://www.threads.net/@arvinsunboy
the problem with threads is - it's so data-hungry that it's actually outlawed here - can't sign up from an European account.
Hi - this doesn't relate directly to AI for conservation, but wanted to share with the group. I got to work with a fantastic PhD student Nataliya R. to create a guide for creating scientific figures including some base fundamentals through a Figure Rubric & Figure Spectrum for evaluating. If you're interested check it out here: https://github.com/nrokh/ScientificFigures
Thanks @Andrew Schulz this is great work by Nataliya R!! Really exciting material. I see that "all items in this workshop are open source and follow... the definition". However, the CC BY-NC license is NOT open source compliant. May I suggest the CC BY-SA license instead? It should also be included in a LICENSE file in the repository which is machine-readable by GitHub, instructions here.
These are fantastic! Thank you for making this!
What is the geographical HQ of AI for conservation or Tech for Nature/Conservation?
The way one "knows" if you want to do Fashion, you go to Paris, money New York, and tech the Silicon Valley. Is there a place that would be home for people who'd want to live surrounded by "likeminded" people.
Thank you : D
I think more dispersed now with so many orgs and companies now remote and WFH becoming more commonplace
As @Jason Holmberg (Wild Me) discovered, if you google State of Conservation Tech, Maryland comes up 😂 maybe a sign we should all convene there
Hi all,
I'm an assistant professor in computer science at UofT, specializing in robotics and machine learning.
I recently bought an autonomous boat that has a programmable winch, allowing
us to move sensors vertically along the water column:
https://www.maritimerobotics.com/otter
I'd like to focus on what sensors it would make sense to deploy on this boat, so
I am asking people who are interested in aquatic and marine environments
for advice on what's worth measuring to advance their research.
Would something like this multiparameter sonde work for you or for people in your area?
If not, do you have any other sensors that you would like to deploy? I've also looked at
RBR sensors for real-time measurements.
I'm currently looking into underwater hyperspectral cameras, as well as
flow cytometers for plankton or microplastics.
Ideally, these sensors would allow for in-situ analysis, but I'm open to any
suggestions that might make this robotic platform useful for scientists
in your research area.
Thanks,
Florian
Hi @Florian Shkurti, have you considered an ADCP? https://www.whoi.edu/what-we-do/explore/instruments/instruments-sensors-samplers/acoustic-doppler-current-profiler-adcp/
I know that some researchers are using these to measure frazil ice concentration, so I suspect they may also work for plankton and plastics.
Thanks so much @Cody Kupferschmidt, I'll take a more detailed look. Do you know the names of the researchers who are using this for ice concentration?
You can take a look at the two researchers below. Let me know if you have any more questions!
https://www.fsg.ulaval.ca/departements/professeurs/tadros-ghobrial-901
https://apps.ualberta.ca/directory/person/mrloewen
Do you have habitats and/or field sites in mind?
Here are some rambling thoughts:
- I work mainly on studying the health of coral reefs using acoustics and ML. We collect audio with Soundtraps and now mostly Hydromoths, plenty of other hydrophones out there too.
- One of my projects uses a CNN trained on hydromoth recordings to detect bomb fishing. We are working in collaboration with SFB who have started putting hydrophones on blue boats (similar to the Otter it would seem). This is to try and allow roaming detection of bombing events https://www.sfbusa.org/
- We also work at the worlds largest reef restoration program which is a pretty great place to do science. We often collect images for photogrammetry and benthic surveys using divers, but as this is so shallow (~2m) I've been aware things like the Otter could do this but better. A field season is now being planned for 2024.
- For this same purpose, hyperspectral images would be fantastic. These have been shown to improve bleaching detection/coral ID, and I would love to play with chucking those extra channels into a CNN and seeing how it improves benthic classification.
If any of that is of interest, do please contact me on ben.williams.20@ucl.ac.uk
Yikes, as of last week we've officially hit 2°C of global warming
https://twitter.com/CopernicusECMWF/status/1726578518463816078
I received an invitation to sign this open letter: Scientist Rebellion Call for Climate Action
Hi all - just wanted to share an update to the GitHub on Scientific Figures (https://github.com/nrokh/ScientificFigures). We uploaded all of our teaching material including slides, lesson plans, and the syllabus! We hope these are helpful! If you have any questions or comments please let me know!
This so cool! I particularly loved this slide
That slide is all Nataliya! Cannot take credit!
https://x.com/natrokh?s=21&t=Ey8UHttbAyUBB0t5Ue8hZw
Does anyone in the community happen to have a connection to these authors? Trying to get in touch about their very rare set of tree surveys from Central Africa in DRC. https://www.sciencedirect.com/science/article/pii/S0378112723007879
I would try Adeline Fayolle at her email address -she replied to us a few years ago about another data set she was involved in.
Ultralytics YOLOv8 Turns One: A Year of Breakthroughs and Innovations
What especially sounds interesting is the YOLOv8.1 release with Oriented Bounding Box feature 🔥!
Hi everyone! NxyCarbon is hosting a sustainable finance event at the end of the month. We’re diving deep into demand signals for ecosystem restoration, biodiversity, and carbon removal, while going over all the ways to finance a project in these areas.
🎈Register here to join the conversation https://lu.ma/nyxcarbon1 🎈
The carbon market has been around for a couple of decades, and has thus far played a key role in sustainable finance, but will it continue amid policy change and calls for greater transparency? Are there other, better, ways to finance carbon removal and ecosystem restoration? Listen to industry veterans chat about the macro and micro forces driving demand for environmental services.
The conversation will be led by:
Brittany Salas, CEO & Founder, NyxCarbon
Nate Wyne CEO & Co-founder, Floodlight
Randy Wilgis COO, Earth Recovery Partners
🔐 NyxCarbon is a lending platform that makes it easy for financial institutions to provide loans to SMEs in carbon removal and ecosystem restoration. The end-to-end solution reduces the cost of underwriting providing targeted market intelligence and automating loan servicing.
🔍 Floodlight is a geospatial data intelligence company providing an extensive catalog of Environmental, Social, and Governance Datasets by leveraging ground sensors, aerial satellites, and artificial intelligence.
🌲 Earth Recovery Partners mobilizes enterprises and projects to solve the world’s environmental problems through nature-based solutions, environmental markets, and technology.
Shir, if you're doing ocean specific stuff you might find MBARI's Video Annotation and Reference System helpful. It has been developed over the past 20 some odd years and is what the institution uses to organizes its (extensive) collection of ROV videos. Supports domain specific metadata, etc.
Just a thought. Depending on the nature of your video data, you could even analyze them using AI algorithms for generating additional info like relevant keywords. This could be added tot he meta-data as well.
I’ll be at AAAI Thursday, Feb 22 - Sunday, Feb 25, with my colleague, Earth Species Project Interim AI Director, Olivier Pietquin. Happy to say hello and connect IRL.
Dear all users of robots/drones for ecology/conservation!
We are excited to announce a new workshop titled "Robots for Understanding Natural Ecosystems" that we're hosting at the upcoming ICRA 2024, in Yokohama, Japan on May 17th!
The workshop invites roboticists who work in ecology or ecologists who work with robots to come and share their findings and discuss the future of these technologies and their role in ecology and conservation topics!
To find out more visit the website: https://warp.whoi.edu/rune or feel free to reach out to me on this slack!
Best,
-ICRA 2024 RUNE Organizers
I definitely won't be able to attend, but I'd love to read a synthesis paper if you all write up the themes of the workshop!
Dear AI for Conservation community!
My name is Xiaojuan, and I’m a research scientist at Climate Change AI (CCAI). I’m reaching out to see if any of you would be open to participating in either a 60-minute group virtual workshop session or a one-on-one interview focused on critical data gaps in using machine learning for work in biodiversity/ecosystems. CCAI, partnering with Google DeepMind, is conducting a global stocktake of critical data gaps that inhibit effective machine learning (ML) solutions to climate-related challenges. The goal of this initiative is to report on the most important data gaps, and lay out pathways for funders, data providers, and researchers to address them.
If you’re interested in joining the group workshop, may I ask you to indicate your availability in the following poll (by March 15): http://whenisgood.net/3xpsa9c?
If you’re interested in participating in a one-on-one interview, please book a time here: https://calendar.app.google/RbDx9GkeDiVxBYJA8
Thank you, and we hope you can join us!
Kinda naive question, I was reading a super cool article about the Ute tribe building a giant solar field in the desert. To site the plant they had to survey the area and find/catalog an endangered cactus species. Is that type of work done by people here or is that more of a commercial/surveying endeavor that doesn’t really overlap with this Slack?
there are at least a few folks on here who do forest mapping using cv; so i suspect this falls within the same category.
I think the techniques seem similar, but I was more curious if the actual process is similar, since I’m under the impression that for something like this an environmental consulting firm is contracted to do the actual work. This Slack seems much more academia and non-profit focused although that might be just a bias of the limited contact I’ve had with folks so far.
Hi Nate, the lab of Brian Todd, my PhD advisor did a lot of related work on desert tortoises in the Mojave. My impression is that for both sensitive species discovery and management, developers often rely on environmental consultants. There is massive time and financial costs to these endeavors. In the case of the desert tortoise, which is listed by the Endangered Species Act as a threatened species, my understanding is developers were given strict quotas regarding the number of animals that could be harmed incidentally during the development process. As a result, one of the articles I’m linking mentions that BrightSource spent >$50 million to try to mitigate harms to the desert tortoise during their solar development of the Ivanpah Valley! This included millions of dollars of fencing to keep tortoises from getting hit on roads. But then the tortoises encounter the fences, sometimes pace them in frustration, and die of thermal exposure as a result. To counteract that, my understanding was that they hired biologists/consultants to literally walk the fences every day, intervening with any pacing tortoises they would encounter. A real mess!
TLDR: solar development has complex environmental impacts and right now mitigation of those impacts is extremely costly in terms of time and money
Some reading and a really good video on the subject:
https://www.latimes.com/archives/la-xpm-2012-mar-04-la-me-solar-tortoise-20120304-story.html
https://www.ucdavis.edu/news/desert-tortoises-can%E2%80%99t-take-heat-roadside-fencing
https://www.youtube.com/watch?v=4TINHeMuwg8
I gave an interview on the MLOps podcast and spoke about the unique challenges in Environmental AI and some of the lessons we have learned building AI for conservation and sustainability. This is not intended to be shameless self promotion (although I know it may come across that way), but I do hope that it may be useful to some of the teams here, and of course I am very grateful for any feedback on the ideas/how we might grow/do a better job (technical or otherwise). https://podcasts.apple.com/us/podcast/beyond-agi-can-ai-help-save-the-planet-patrick-beukema-225/id1505372978?i=1000653011203
Beyond AGI, Can AI Help Save the Planet? // Patrick Beukema // #225
If anyone else had legacy data in which annotators had counted items in photoshop, I managed to figure out how to export the locations of the markers https://github.com/weecology/EvergladesTools/blob/main/photoshop/counts.jsx and I made a video (that was meant for internal use, sorry i'm talking fast to the head of my field team). You do then need to fill in the labels in a csv file. Not bad for recovering very old data. https://www.loom.com/share/5ac181c1a8884194825766589c9642d5
<a href="https://github.com/weecology/EvergladesTools">weecology/EvergladesTools</a>
If yes, I'd be happy to join too! So great to learn there are other AI for Conservation people here. 🙌
Hi!
I wanted to share an interesting knowledge distillation approach that leverages LLMs to label image datasets and then train much smaller and constrained ML models that can fit in camera traps or other edge devices
https://youtu.be/Jou0aRgGiis?si=B81TKs1G-6ZBx1jW
Does anyone have experience with ecological data and object detection methods that explicitly use pixels and context from outside the prediction detection/anchor box to inform detection classification? Is this one of the places that the YOLO type models differ? I feel like I should know the answer to this. Clearly important for animals that occur in groups. I'm reading about box attention here https://arxiv.org/pdf/2111.13087. Maybe @Nate Harada or @Oisin Mac Aodha. There are lots of post-processing methods like conditional random fields(@Benjamin Kellenberger’s paper here: https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1002/rse2.200), but I'd like to learn the spatial attention. Does anyone think this is worth the effort?
I believe the features in YOLO-style models should have context from outside the box, even after region cropping in feature space. But that context will be stronger from near than far. This was one of the things we were trying to capture in a more object-centric way with the cross-box attention on context r-cnn, but that's pretty old school at this point
We have some experiments in this paper (Table 2) where we should the benefit of context from outside the region of interest when detecting bat calls in audio:
https://www.biorxiv.org/content/10.1101/2022.12.14.520490v1.full.pdf+html
You can also check out some experiments we did here on this. For YOLO I think this should be learned implicitly, but idt I have seen anyone do a SHAP-esque analysis to get an understanding of how those features are helping/hurting.
I don't know how to quantify "explicitly", but YOLOv5 definitely weights nearby pixels a lot, even compared to pixels inside the box. E.g. sometimes it seems really impressive that MegaDetector finds people in the windows of cars, but it's a parlor trick; MD also hallucinates a lot of people in car windows, it's just learned what car windows look like, and that somewhere inside the window (but not taking up the whole window), there is often a person.
Do you think there is something particular about the yolo style models that are more likely to learn these kind of context features compared to torchvision style retinanet/maskRCNN or the vision transformers?
SSD Multibox and related networks uses convolution layers above the classification - so there is context information based on the convolution kernel size outside of the bounding box but with a limited range - this gets more interesting when using "feature fusion SSD"s since the network gathers data from the whole image in the top layer and than feeds it back to the detection layer through deconvolutions - via that route the network can take information from the whole image into account for localization and classification of any indivudual object regardless of size -
just checked according to the architecture here: https://iq.opengenus.org/yolov5/ yolo5 does exactly that - it pools information from the whole image through the neck - so in principle - if the training data supports that - far spacial relationships can be taken into account
Somewhat related https://arxiv.org/abs/1612.04402 - a good discussion on improving tiny object detection with varying sized context windows. Not sure how relevant it is nowadays with a lot of models using FPN to combine features from different layers
Hi all, this workshop I am organizing is not specifically for conservation folks, but we'd love to have anyone with an interest in the topic (and no experience necessary). Here's the call: Are you interested in attending a workshop in Ashland, Oregon in September to learn how to build your own RAG/LLM? Please apply and/or share with anyone interested in this topic. https://ragworkshop2024.github.io/ Feel free to reach out if you have any questions! Travel support is available, for flights/accommodation (or other transpo). Come visit beautiful Ashland!
Hi all. Anyone have experience with point cloud (LiDAR) classification? What are the current go-to utilities for labelling such data and go-to architectures? ☁️
A bit scale dependent. Are you most interested in satellite (like ICESAT-2), airborne (like https://usgs.entwine.io/) or ground vehicle collected?
Unfortunately I haven't worked with LiDAR in an ML context. What is your application? For some tasks like ground plane estimation you may be able to find conventional approaches that work well, such as those implemented in PDAL.
Def agree sometimes the "boring" stuff does just fine... plane fitting, DBSCAN, K-means, KNN, depending on what you need. For fancier stuff: https://github.com/torch-points3d/torch-points3d
I’m curious about both types of tasks, but mostly looking at drone capture of man-made structures (e.g: utility poles/towers)
Do you have imagery or just LiDAR? Shameless self promotion for a tool that I'm working on for doing semantic reasoning with multiview drone imagery: https://github.com/open-forest-observatory/geograypher
But, FWIW, the imagery is taken by flying around the structure, trying to capture all sides. Not from above.
I think you might have good luck using image-based detection/segmentation for that task. I could also imagine a variety of ML-free approaches to the task using point clouds.
Leveraging side views is a big part of why we developed geograypher, that's not a problem.
Awesome, feel free to open an issue or reach out here if you have any questions. I'm also happy to set up a time to chat. We'll have a docs website up in the next few days.
I worked on lidar ML at both cruise and waymo if you’re using ground or low level, for high level aerial I’m not as up to date. Feel free to DM me happy to chat more!
I've been reading https://www.researchsquare.com/article/rs-3832815/v1 Has anyone spent time on a 'species not in the class' alert system for human review, many of the zero shot tools from outside ecology and anomaly detection ideas end up finding 1) smeared or non-recognizable detections, 2) non-biological objects that were incorrectly detected. I'm thinking about ways that we can find new bird species in UAV imagery after first using a 'bird' detector and classifying each crop. When the model believes that it has found something of interest, and it reaches a reasonable objectness and salience threshold, send to humans to look at. Maybe @Eric Orenstein and fathomnet team. General CV review here: https://link.springer.com/article/10.1007/s11263-024-02117-4
We have also been (super loosely) thinking about this type of thing, and I think generally agree with the multi-stage style approach. Basically the most-specific object detector/classifier you can arrive at "above" the target classifier (MegaDetector's a good start, bird detector if you have that, etc.) and then whatever level it is unable to produce a reasonable confidence threshold, raise that as something needing humans to review. If you have lots of data in this manner, you can attempt some flavor of unsupervised clustering to see if you can find some groups that might be a missing species or some other systemic issue.
I did something similar for individual re-ID on dolphins during my PhD, so same species but the methods could apply I believe.
The problem of classification was open-set, as there was no guarantee all individuals had been previously ID'd and were in the catalogue. We trained a Siamese Neural Network on examples of all known individuals. Once trained, examples of the same individual would generate similar embeddings which could then be clustered. Previously unseen individuals would in theory generate dissimilar embeddings, and based on some distance threshold could be flagged for manual review. Generating new clusters could be done by storing the new embeddings in some database for future comparison, though due to covid we couldn't do further fieldwork to expand the catalogue and test how effective that was.
The problem is central to my current project, detecting taxa in Antarctic Southern Ocean sea floor imagery. Here there is a much higher chance of the data containing new species (both to our dataset and science), so having a robust way to flag potential new species is key. Very keen to discuss work in this area.
Interesting stuff. I have recently attempted to do something related (haven't had much success but still working on it!) where we attempt to discover ``new" species currently miscategorised into existing classes using online, community collected observations of those species (iNaturalist basically).
I also went with a "learn useful features for classification into existing classes -> look for clusters in this feature space -> use some metric of distance or dissimilarity between clusters to identify potentially new species" approach, possibly similar to yours @Cameron Trotter
Anyway I need to look into it but maybe these beetle datasets will be helpful. I still have a lot of stuff in this area to explore for now.
My first pass at these approaches is that they are excellent in finding very different objects than target classes, so they are good enough for data cleaning. For example from the UAV birds, when we've detected a distant boat instead of a bird, but they are too coarse to find the fine-grained differences that differentiate species. We need some kind of salience based fine-grained detector,
Yeah, I would say that so far the most common "new species" I identify are just cases where photos show evidence of the presence of a species in a form that isn't actually an image of the animal (such as nests, scat, etc).
@Nico Lang has a really nice paper formalizing what Ben just said: https://openaccess.thecvf.com/content/CVPR2024/html/Lang_From_Coarse_to_Fine-Grained_Open-Set_Recognition_CVPR_2024_paper.html
Just read, @Nico Lang that's a great and well written paper. For those in the thread that haven't looked at the paper, one interesting takeaway is that hierarchical models, which have been shown to improve within set accuracy and 'make better mistakes', actively hinder novel class detection by increasing the similarity of species, for example within a genus. By discouraging hierarchical structure, you get a small, but meaningful bump in novel class detection. This result, coupled with another result showing the complexities of label granularity for the supervised classes underlines how hard its going to be develop a single model that can be used to both classify species AND recognize novel ones. This feels like a problem that will need to be addressed in two separate stages. @Nico Lang jump in if I misinterpreted anything. If you have any thoughts on where to continue to take this, feel free to throw out ideas.
My lab and I are super interested in human-AI collaboration in general, and I’m not sure we’ve looked at this exactly, but we have two papers where we’ve started thinking about related ideas (the more recent one with @Sara Beery and @Vaibhav Balloli), if maybe helpful: https://arxiv.org/abs/2112.06751, https://realize-lab.github.io/CHAIR/static/pdfs/IJCAI_CHAIR_CameraReady.pdf
Really great @Elizabeth Bondi-Kelly, I like the general narrative alot. On a similar theme, there is alot of statistical work to be done to think about how to sample large image datasets for human-AI collaboration. Imagine you are working with a management agency that wants 90% accuracy for species classification and animal counts in 1.5 million images (as we are), and you don't want human review of all images. How to develop bayesian type models to estimate the likelihood that pipeline, rather than the model, has a certain accuracy and therefore the mission is complete. Given the nonuniform distribution of classes, how/when can we say that the set of images reaches as certain performance threshold given limited capacity for human-review. It seems pretty safe to say that random sampling won't do it, it will need to be some kind of embeddings informed sampling. Related to the model ideas above.
Definitely - thank you! Let me know if you'd like to chat more, we'd love to follow up 🙂
https://arxiv.org/pdf/2407.08908 Updated paper link just in case anyone was not able to access our IJCAI paper
Hi all, does anyone know how to access the GPS coordinates for the iWildCam camera trap images? I get unauthorized error when I try to download WCS annotations, obfuscated GPS coordinates and MegaDetector v4 detections from the official page https://github.com/visipedia/iwildcam_comp?tab=readme-ov-file (or if I try to access any of the data links)
Hi Emmanuel, we've been trying to sort this out for a while now. The file got deleted on accident after some data got moved around at Microsoft recently. We're now following up with people to see if anyone has a local version (so if anyone does, let us know!)
All of those iWildCam files should still be accessible through the Kaggle competition page: https://www.kaggle.com/c/iwildcam2022-fgvc9/data. @Emmanuel Azuh, take a look at let us know if anything is still missing.
@Sara Beery I seem to have a local copy of all files under https://github.com/visipedia/iwildcam_comp#data-downloads. Let me know where you want to upload them.
All the files are on kaggle except the GPS file, is that there?
That's also there. What's not on Kaggle are the additional data sources we provided in the competition: remote sensing data, iNaturalist 2017 images and annotations, and iNaturalist 2018 images and annotations. But I have a copy of those locally.
I'm attaching the GPS locations file.
Awesome. @Timm Haucke any chance you could work with Stefan to get a backup on our MIT server? And then I'll follow up about long term hosting.
Absolutely. @Ștefan Istrate I’ll follow up via PM. @Sara Beery we also have the option to expose datasets hosted on MITs servers via HTTP, so that might work at least as a medium term solution
I'm happy to host on LILA... I don't normally host re-slicings of other datasets, but iWildCam is a special case since it's taken on the role of "closest thing we have to a standard benchmark". This also might be a good opportunity to clean up some of the label noise; at least cleaning up some of the animal/empty confusion would be relatively quick. We can also definitely update the MD results to MDv5.
Thank a ton all for the quick response! I downloaded the gps_locations shared above 🙂
Thanks!! I've also been trying to access these for a bit
After this thread reminded us that some of the iWildCam data had gotten fragmented, @Timm Haucke gathered all the iWildCam2022 data+documentation, and we posted it as a dataset on LILA:
https://lila.science/datasets/iwildcam-2022/
Thanks, @Emmanuel Azuh, for starting this thread!
Oops I almost forgot to include a gratuitous camera trap image from the dataset:
Hi @Dan Morris and @Timm Haucke many thanks for updating iwildcam 22 page on LILA! Although I got access to the 2021 competition GPS info in the slack channel, some further lit review led me to use the wilds version (with iwildcam 2020). I was wondering if the same locations were used between 2020/2021? If not, it would be great to access the gps md (still unaccessible to me from the github link in the LILA page for prior years or 22 competition https://lilablobssc.blob.core.windows.net/iwildcam/iwildcam-2022-metadata.zip )
@Ștefan Istrate @Elijah Cole (Deactivated) @Sara Beery do you know ☝️?
The first part of your question makes it seem like you're looking for the location data for iWildCam 2020, but the last part makes it seem like you're having trouble finding the location data for iWildCam 2022.
I don't know the answer to the first question (re: location consistency across years), but in terms of accessing the GPS locations for iWildCam 2022, the iWildCam 2022 zipfile (which you can download from LILA) contains the file "metadata/gps_locations.json". Is that not what you're looking for wrt 2022 data?
(And although I can't help with the question of location consistency... if the location identifiers in the 2022 .json file match those from the 2020 image .json, it seems extremely likely that they are the same locations.)
It should be the same data with a different task, classification vs counting.
2020 is classification, 2021 and 2022 add counting
I am looking for 2020 yes but can't down github md for any year so can't really cross check with the 2021 version I have.. @Sara Beery there are overlaps so I assume the missing data are what belong to the same country?
this is the 2021 md locations I have gps info for
this is the wilds (iwildcam 2020) train locations
We coudn't release locations for all the data due to restrictions from some data providers, this is outlined in the white paper for the competition
I wanted to double check the restricted set corresponds to the missing data in the figures sent above because the missing set is not consecutive numbers..
The gps location file linked is the only one that was released, we don't have additional locations with public gps.
Hello @Timm Haucke if you don't mind, I have another question regarding the iwildcam dataset.. I've been trying to get a mapping from the competition categories to inaturalist categories but haven't managed to find a complete mapping so far. For instance the inat2017 competition page provides a mapping from inatIDs/GBIF IDs to inat2017 categories but their entries are not complete for either of those. When I try to use the actual species names as well, they don't overlap with the scientific_names in the WCS metadata. The iwildcam competition mentions there's a mapping somewhere which I'm guessing is WCS annotation at this link but so far, I still get unauthorized when I try to download from the links on the iwildcam github page.. Would you happen to have an idea how to get the mapping?
Hi @Emmanuel Azuh, AFAIK the WCS annotation files should be included in the big zipfile download on the LILA page. @Ștefan Istrate, do you know where to find the taxonomy mapping? Or is the mapping just directory names under inaturalist_20** matching category names in the COCO annotation files?
Ohh is that the only way? I haven't downloaded the full data but using the subset from wilds python package..
After investigating this further I think we're indeed missing the taxonomy mapping files, which are named inaturalist2017toiwildcamtrain.json and inaturalist2018toiwildcamtrain.json . I'll try to find them and let you know
@Emmanuel Azuh: I'm attaching the 2 missing taxonomy mapping files, just to unblock you until Timm puts them in a more permanent location.
@Emmanuel Azuh We did a little digging to make sure everything made its way to the copy of record on LILA, and it looks like these two files were already there, inside the "metadata" folder, although they were not mentioned in the README or on the LILA page. We've added them to the LILA page and will update the README within the zipfile, but just to make sure we're parking our cars in the same garage, can you confirm that one of the following things is true... either (1) you got your copy of the data from somewhere other than the LILA page for this dataset, or (2) those two files are there inside the metadata folder in your copy of the dataset?
Sorry for not documenting this, I think this dataset is almost at steady state. Thanks for helping us get this all sorted out!
Hi @Dan Morris, I ended up downloading the wilds iwildcam using this paper's github. It's a smaller subset of the full dataset and allows for easy comparison.. I didn't end up downloading the full zip from LILA but I also couldn't access any of the other links from the iwildcam github (metadata etc), except the copies I got from this slack..
@Emmanuel Azuh all the links that were previously mentioned in the 2022 iWildCam README are dead, but all the files should match 1-to-1 with the ones on the big zipfile available on LILA
@Emmanuel Azuh thanks, all clear, and everything sounds right there, the universe is back in order. The LILA copy is now the copy of record and the one we believe is complete; if anyone sees anything missing from that copy, let us know. I've updated the README within the LILA zipfile to refer to these mapping files, and AFAIK everything is now at steady state.
Cool this sounds great! If I run into any issue, I'll download the full zip from LILA then 🙂
Let's play a game... how many camera trap images do we think have ever been collected? Let's leave video out of it for now to avoid questions about whether a frame counts as an image. Bonus points for guessing how many camera trap images were collected in the last year (presumably camera trap image volume is increasing with time).
I wish there were a way to decide who wins this game, but I have no idea what the answer is. So I'm not sure whether it's even really a game. I guess no one wins at Minecraft either, and Minecraft is a game.
There is no secret motivation behind this game, I was just inspired by the #random channel.
I have a guess for both questions, but I'll hold my guesses until others have guessed, it would be lame if I posted the question and made the first guess.
fun game! guessing based on order of magnitude based on my analysis of Wildlife Insights...
total ~ 1B
this year ~ 10 million
How about the number of publications on Scopus that mention "Camera trap" (5,457) multiplied by some arbitrarily chosen multiple (200,000) = 1,091,400,000 camera trap images in total. And for the last year I will guess that many of the existing camera traps are still collecting images and many studies are going to be publishing data collected across multiple years so I will use a new multiple (140,000), and there might be around ~824 studies this year based on the equation my Excel plot fit to the number of studies per year so 824**140,000 = 115,360,000. I think I am completely off on the second number... 😅📸
If we got 1M in Toronto from 33 cameras with continuous sampling of 1 year. Based on this we’d only need 10 urban studies to reach 10M this year 😅
OK my guesses... the one variable I can put some bounds on is "number of images processed with AI every year". Let's call that NAI. I can put a pretty tight lower bound on NAI for recent years at ~200M, and a looser upper bound at ~400M. I'm sure that's growing, but it's roughly consistent over the last couple of years, I think, at least within an order of magnitude. That also doesn't mean the images processed with AI in, e.g., 2023 were collected in 2023 (they weren't), but I generally don't think it's the case that latency is changing a ton over time.
I can't really even try to estimate the percentage of total images that end up being processed with AI, but I've tried (mostly unsuccessfully) to estimate the number of ecologists/projects using AI, via the time-tested method of "ask random people at ecology conferences whether they use AI". At steady state, publications will be a very good indicator for this ratio, but I think we're still within publication latency of almost anyone using AI, so I don't think the publication record will be a good indicator of this ratio until maybe 2026. But FWIW, I put this number at maybe 5%-10%. Those tend to be high-volume users, so I think maybe 20% of all images collected recently end up being processed with AI. So I would guess maybe NAI/0.2 == between 1B and 2B images are collected per year for conservation and ecology.
If I search Amazon for consumer camera traps, and look at the rough number of products and reviews, and read random Web pages about the percentage of people who write reviews, I'm guessing that the number of consumer camera traps (usually "trail camera" in consumer parlance) that are out in the universe is somewhere in the low tens of millions, heavily biased toward recent years. My guess is that most of those are used briefly and discarded after a few thousand images, but maybe that's another 1B images per year.
So, my guess is 2B-3B images per year in the last couple of years.
And I would guess that with falling storage costs and the availability of consumer cameras, image rates are growing so fast that everything before ~10 years ago is probably negligible in the total count. So I'm guessing maybe 20B camera trap images exist in the universe.
Have there been efforts to dig up historic camera trap data and preserve it? Blown away that camera trap studies have been happing since the 90s (or earlier?)? I asked an author for an image from a 2003 paper. They spent some time looking, but alas images were lost. If we don't ask now we may lose some of the early stuff forever.
https://en.wikipedia.org/wiki/Fermi_problem 🙂
While I was at ACGI with many great people from this channel (nice job @Sara Beery and David for organizing), one of the needs that came up from land managers in the West is livestock detection on public lands. I thought this would exist already. Does anyone know of an airborne cattle/horse detector. Quick google searches shows plenty of papers (https://www.tandfonline.com/doi/pdf/10.1080/01431161.2019.1624858, and 20+ others https://scholar.google.com/scholar?start=20&q=livestock+detection+object+detection+uav&hl=en&as_sdt=0,38), but no open source implementation that anything like a non-researcher would use. I'm going to just follow this with one of my students for a couple days if anyone wants to get involved, I write each of those authors and try to organize the data. Then it would be interesting to apply to NAIP data in the west and see what happens. If anyone has experience in large scale NAIP processing, I welcome ideas on how best to do it.
@Ben Weinstein I’m very interested in this same use case for Africa. Cattle management is a massive problem in Tanzania and many other countries. Managers have only a vague idea of how many there are and where. I am not aware of any options. I’d be interested in knowing about anything you learn or pursue.
This might be interesting, though it seems to be very low altitude: https://research-information.bris.ac.uk/en/datasets/aerialcattle2017
For fun, I emailed the first 20 authors on that google scholar hit. 5 emails were dead so far. Thanks @Timm Haucke.
This feels like the first thing that @Ben Weinstein would try, but... my guess is that you will get not-random-but-not-quite-useful results from HerdNet (which is great work, but does not have a domestic cattle class AFAIK):
https://github.com/Alexandre-Delplanque/HerdNet
...so maybe fine-tuning HerdNet on data from WAID:
https://github.com/xiaohuicui/WAID/
...would give you even-less-random-and-closer-to-useful results?
I know you asked for a model and I gave you random starting points for creating a new model. My bad.
no thats great. I remember talking to @Alex Delplanque that those data were not open, but we can always pass HERDNET over some unlabeled images
The HerdNet training images are not open AFAIK, but they also don't have cows, so it probably wouldn't solve your problem anyway. AFAIK the WAID data is the closest open dataset to what I'm imagining you're looking for.
k, i found 7 dataset so far. Also, amazingly, I believe I found an author who over a period of years intentionally misspelled their correspondence email by dropping a letter from their name. Presumably to avoid scrapping/being contacted? Never seen that before. My first emails bounced and when I changed it to his correctly spelled name it went through. 5+ pubs.
Definitely interested in getting cattle/horses into our models. Please keep me in the loop on this.
Thanks Jason. Sorry you didn't end up in Aspen, I saw you originally listed, was a nice meeting, this grew out of a conversation we had with department of interior.
Here are the public datasets I found today. dropbox still uploading https://www.dropbox.com/scl/fo/ugqb64gqbiuej7ordg3tz/AK9JrXq37Mid7_vTaG7sagM?rlkey=su6jp73bi1fapctl4izvox4cw&dl=0. If anyone wants to play along.
Of course I want to play along... any chance you have links for them? Some I can find from the filename, but, e.g. "images.zip" is going to be hard to reverse-engineer.
Yes, sorry, it’s not that I named them, that’s what they were called. I was going to collate them into one thing and label them from where they were from. I was hoping they had a readme in each of them. You are out front of me.
@Howard L Frederick please take note of this thread. I know this is a problem you have wanted to tackle in Tanzania for ever.
@Ben Weinstein I definitely know you didn't name them, I've seen your datasets and they have fantastically clear filenames. 🙂 LMK if I can help with any of the detective work!
@Lasha Otarashvili Data, glorious data!
Starting to put together a list with links and renaming the files. If anyone wants to help sort through these and see if they look usable. https://docs.google.com/spreadsheets/d/19ybt0Hl52mY-DyEvPkaQxcaEeC2QrsQLuw2cOF7C6W0/edit?usp=sharing. Most appear to have a readme when you unzip.
I have a prototype of a livestock detector. Still need more evaluation data if anyone else knows where to get it, I'd like to test zero-shot performance.
You could ping our community ben, see if there is data that comes back? Try a discussion here - https://wildlabs.net/groups/ai-conservation
awesome - we'll also give it a shout out next variety hour (last wednesday of the month) and see if anyone has anything to share. Realise you're on a role and thats a little while away, but might help!
@Ben Weinstein we have rather a lot of nadir and oblique livestock imagery for Tanzania (mostly from systematic flights around protected areas / boundaries), but can’t share publicly (yet); I’d be keen to work on this with you?
sure, would you be okay with us hosting a livestock model that was derived or validation from these data. I have no real plans on publishing this as a manuscript, just trying to do service to the community, so expectations for reproducibility are definitely lower.
@Ben Weinstein we have cattle dataset from manned aircraft done in Kenya but yet to make data public at the moment. I am interested in the idea https://doi.org/10.3390/rs16162929
Great, I’m away for the weekend, but be back on Monday. As I’ve said to others, this is very much just a service for the community, blog post kind of idea. And so I’m happy to follow whatever data restrictions you need. We can always circle back and talk about authorship, etc., if this ever turns into something more proper. At this point, I doubt it.
@Ben Weinstein I've been meaning to go through your list of aerial livestock datasets (from earlier on this thread) and add any that I'm missing to my aerial/drone dataset list (I can take care of thumbnails, sample code, etc.). Is the DropBox link you shared earlier on this thread (a) complete (in terms of the datasets you found and used for your prototype) and (b) stable (in the sense that you don't feel like you're about to add ten more)? If so, it looks like there's enough information there for me to find all the sources.
@Ben Weinstein I'm in the process of building a dataset of high res aerial imagery collected throughout Wyoming with annotated cattle and pronghorn. The current images are property of the state so may not be public for a while but I think could be shared for internal testing purposes.
Does anyone here live in Juneau, AK? I just moved here and don’t know anyone. Would be up to go look at birds some time~
is this coyote carrying a brick?
oh nooo, a whole loaf of bread!!!
Hi! this is from a dataset in Alberta
lol I got the slack group wrong, this is not AI for conservation material but Im glad you get to enjoy this proud coyote, I wonder what CV would recognize that bread as.
how does he stay so svelte on a high carb diet like that? le sigh
Everyone can appreciate a tasty loaf of bread
Does anyone know of python module for automating the download of images from iNaturalist open dataset for a list of target taxa? https://github.com/inaturalist/inaturalist-open-data/tree/documentation, should I write up a few tools, or is it better to go through the released computer vision dataset and see if you taxa is in there. That means downloading alot of data. Imagine I have 30 taxa and I want 1000 images for each.
I don't know of anything unfortunately, but it sounds very useful if you do find something!
We have a couple tools for this type of purpose:
Distributed-downloader is larger scale for use on an HPC with MPI. We used it to download the GBIF monthly snapshot as it's geared toward multiple servers and a fast download.
cautious-robot is a simpler downloader designed to just download line by line from a CSV (on a laptop), but you could set it up to be distributed manually.
If you're using metadata from GBIF, I think these would still require downloading rather large zips just to get the URLs unless you did a search to reduce to the taxa of interest.
@Ben Weinstein here's a python script I had chatgpt write that works pretty good.. The script read froms a file named check_species.csv for genus+species name to fetch data for.
@Ben Weinstein https://github.com/pyinat/pyinaturalist/
Is anyone interested in being a podcast guest?
Someone from another community is doing one entitled "A senior, a Junior and I" and talks about different topics each time.
It's a mini podcasts with three episodes so far, but it's a fun way to get started/have the podcast experience/get a different story out.
It is focused on tech in general, but would love tech for nature to feature more so would love the rappresentation from this group!
If anyone of you is interested I can connect you with him (and of course answer any questions before hand.
I think he's thinking of doing the next one on interviewing, but he's doing new topics on a rolling basis, so connecting/being on the roster or even suggesting a topic would be great.
Hi! New to the group and happy to be here 🙂 I've got a question for other BirdNET users: has anyone attempted to adjust BirdNET for sampling rates other than 48 kHz? I'm using BirdNET for transfer learning, and the network I've built on top of it is made entirely from (and will be used on) data sampled at 16 kHz. BirdNET resamples everything to 48 kHz, and I'm wondering if performance might be better if it didn't. From an image perspective with a 0-24 kHz spectrogram, everything from 8-24 kHz is essentially 'blank' in my data- doesn't that impede training compared to a 0-8 kHz image where every pixel contains relevant information? Or should it not matter? And yes, I am aware that spectrograms /= images, just using 'image' as a placeholder. Thanks in advance for any perspectives on this!
BirdNET's spectrogram(s) only go up to 15kHz, despite a Nyquist frequency of 24k. But the spectrogram is hard coded as a layer in the model package so you can't really tinker with it much (unless your going to go deep in teasing apart the saved model files). You could speedup your data by ignoring the sampling rate and feeding it in 160,000 samples at a time, without resampling, but it would shift your target sounds up the frequency scale.
I ended up trying something similar to what the BattyBirdNET team is doing- you can change your sample rate and sample length in the config file, so long as sample rate ** sample length = 144000 (for regular birdNET, 48000**3 = 144000). For my data, this means that I'm now dealing with classifications on 9 second segments instead of 3 second ones. Jury is still out on whether this made much of a performance difference.
Just be aware that doing so is shifting your sounds up the frequency scale and may be pushing them our of the receptive range for BirdNET
👍 👍 performance was essentially the same, in any case, so I don't think I'll be pursuing that particular option any further 🤷♀️
You may want to try a different embedding model to see if a different embedding works better for your target sound. Something like Perch, SurfPerch, or even yamnet/vggish (which both work with audio at a lower sampling rate)
thanks! I would like to, but I'm getting pressure to move forward from my supervisors. So it may be time to just figure out how we can best work with what we have, at this point. It's definitely not bad, just not as perfect as I would've hoped 😛
Over optimizing on your evaluation set might not be ideal anyways. The variance in model performance on different distributions (different sites, years, etc) of real data will probably be larger than any marginal gains your scraping for by finetuning birdnets sampling rate.
yeah, that's very well said
Hi all, are there researchers of public healh, suatainability or policy planning from Mexico region here? I'm looking to connect to learn more about how to implement policy mixes for sustainable development, particularly in the Chiapas region
Finn, in a collaboration between Urban Biodiversity Hub and IUCN Commission on Ecosystem Management we offer Cities & Nature Regional Briefings that would connect you with the people you are looking for. Mexico is part of the Latin America regional group. They meet 6 or more times a year virtually for 30-minute updates. I am happy to connect you :)
Hi everyone! I was wondering, does anyone know of any open-source ungulate detection/segmentation models that use thermal imagery?
Do you need a camera trap model, or an aerial one?
Not open source yet, but I have a bunch of thermal imagery for ungulates that I've been meaning to release for a while (FLIR Duo Pro). Several drone flights over UK landscapes so not likely to transfer to other places - we did some flight trials in Nanyuki, Kenya and found that it's basically impossible outside dawn/dusk because the ground heats up so much. It would be easy to retrain a more recent model as this was quite a long time ago.
Ours is in Maui county. We’re looking for an aerial model - the data we have was taken from a helicopter and the video is pretty shaky, but the animals stand out fairly well from the rest of the background.
@Josh Veitch-Michaelis I would be very interested to hear when those thermal UK drone images become open-source!
Yeah these have been sat on a hard drive for several years, unfortunately COVID got in the way of a lot of plans for further testing. 😅
I was going to refer you to Josh too 🙂 here is the dataset I worked on, taken at various places in Africa: https://lila.science/datasets/conservationdrones. more details here: https://teamcore.seas.harvard.edu/files/teamcore/files/2020_07_teamcore_wacv_birdsai.pdf
@Blair Costelloe and her student will likely have one soonish...
Super last minute, but i have an extra ticket for the blue earth summit if anyone is in London and free today, tmrw and friday.
Awesome just come here, share email and i’ll send you the details!
Hi all, a small advertisement for phd positions in my group. If someone is interested in doing a PhD in AI/robotics for conservation, or knows someone who might be, please share with them this post I made on linkedin: https://www.linkedin.com/posts/ahmadaamir_phd-jobs-imprs-activity-7259859825373728768-0ryS?utm_source=share&utm_medium=member_desktop
Returning to my statistical machine learning discussions with @Toryn Schafer, @Sara Beery , @Casey Youngflesh @Ben Augustine if anyone wants to play along. Are you familiar with any papers on bayesian measures of machine learning evaluation metrics. Imagine a completely unannotated dataset and a previously constructed object detector for a single class. Now I collect 10 test images and measure the true positive rate, the proportion of objects which are detected by the detector. Obviously this evaluation metric is not representative of the entire dataset, which has millions of images. I'm brainstorming a bayesian model that describes the probability distribution that this evaluation metric reflects the true, unobservable, true positive rate. Related to https://arxiv.org/pdf/2103.05331. Ultimately we would construct a workflow that iteratively chooses both train and test samples to annotate to arrive at a desired success rate in the least amount of annotation time. The naive thing to do would be to let the trained model explore the embeddings of the unlabeled test set to choose representative samples, but this would lead to a 'waltz of death' (as @Sara Beery and I just coined on the phone) as biased train chooses biased test, reinforcing each other.
@Sara Beery and I have been working on this recently in the context of model selection, also building off this Active Testing line of work (maybe you already talked about some things we've tried). Would be happy to chat about it sometime. There is a follow up to active testing that does quite a bit better empirically at the expense of having any kind of theoretical guarantees: https://arxiv.org/pdf/2202.06881
We explored something along these lines with Lightly's tools (they have some neat software for self supervised, data sampling and active learning). My experience was it seems to be an approach that works well for some specific examples on paper or toy problems in the literature, but we struggled to get significantly better results versus picking some relevant diversity/sampling measure (like geo location or biome/climate) and going with it. I've also tried some simpler techniques like Cleanlab since most of these tools also aim to identify poor labels, but how to apply them to object detection/segmentation models is often unclear.
The big problem is it takes a lot of experimenting to prove that an active approach made a difference, and if you have to do that then you might as well label everything to start with.
https://www.lightly.ai/
When you say "relevant diversity/sampling measure and going with it", did you use the current model embeddings to sample the remaining images?
I meant in some cases picking a sampling method by gut feel can also work - for example making sure the dataset has imagery from different locations, not over-sampling from a single orthomosaic, etc. Statistical approaches seem to do no worse than random, but including domain knowledge in the process also seems sensible to me.
We tried sampling images from embeddings - Lightly has a bunch of methods for this (for example you train a self supervised model on your big corpus and use that). Though I think the default approach they use is to look at predictions to see where the model is uncertain e.g. mutual information/entropy of predictions. One problem is whether it's necessarily correct to look at an embedding for the overall image if you're interested in potentially very small regions/objects. Maybe this is where the Bayesian approach would work if the process is only driven by the loss value for the image?
https://arxiv.org/abs/2004.04699 < I think they also explored active test set construction
I put some thoughts down as a reminder for myself, no promises of its coherence. https://docs.google.com/document/d/1UxIzqv9JfZ0QwBRADaL87SzgMetYJFJiMNGTT4mBClE/edit?usp=sharing
I think this is another cool, potentially relevant paper https://arxiv.org/abs/2301.09633
and the active follow up : https://arxiv.org/abs/2403.03208
This last paper, skimming quickly, is very similar to what i'm looking for, just with a frequentist, not bayesian perspective. Very helpful. Thanks everyone.
Very interesting stuff! Related but perhaps slightly tangential, does it seem like it would be possible to generate 'group-varying' performance metrics using active testing (or is that already being done)? That is, deriving some latent characteristics from the input data (e.g., image) and generating performance metrics for each of these 'groups' (or performance metrics as a function of some continuously varying set of latent characteristics). Some groups have better performance than others but you might not know what those groups are a priori. If you could derive those latent characteristics for your data easily, you might be able to get at some observation-specific measure of performance which could easily be incorporated in downstream (Bayesian) models.
@Justin Kitzes, this discussion might be of interest.
Anybody has experience using PrimNet or LemurFaceID? A friend, former colleague, wrote about a PhD student of theirs trying to distinguish between Mouse Lemurs, their faces are very similar to to other lemurs, just smaller, which shouldn't be an issue for CV since she has closeup images, so Im thinking she could tryout one of the FaceID codes for primates, so I dug a little to help them out, and the paper describing LemurFaceID shared the dataset they use but I couldnt find their code nor a reference to the algorithm they use, I found the code for PrimNet from a paper citing them but there is no readme https://github.com/harora/Primnet-pytorch.
Hi Tiziana, are you thinking about doing the re-ID for the Mouse Lemurs? Why don’t you try MiewID? https://aiforconservation.slack.com/archives/CLWGQ4BJ6/p1730387852031829. Also, you have a Slack channel about animal_re-id.
Hi Antooon! How are you doing!! Yes that’s exactly so! Thank you so much! I’ll check it out MiewID, student has little experience with ML but just has 100-200 photos to go through
MiewID v3 included LemurFaceID with permission from Professor Jain from Michigan State.
As well as several primate catalogs
that's perfect, so based on that, I'm confident MiewID v3 should work on the mouse lemurs. 2 follow up questions: Has anybody made a tutorial or blogpost, on how to run the MiewID on a folder of pictures to try out if it works? and Do they need to crop, rotate and annotate the eye position as pre-processing?
We put detectors in front of MiewID in Wildbook to crop down to the face. No further annotation is needed assuming the faces are not upside down or severely rotated, etc.
cool! should work! the code is not specific for day pictures right?
LemurFaces were largely well lit day photos if I remember the training data correctly. Across MiewID there are day and night images from camera traps, but I would be surprised if we didn't have a heavy skew toward day time imagery.
oof, ok these are night pictures, but they may be up for sharing the dataset of mouse lemurs to include in MiewID, I dont think they will care if they get help with this re-id they need, seems like it is a big limiting factor for this phd thesis right now. I would suggest to them that they try running MiewID on their photos and then decide what to do next based on how it works.
Hi everyone! Please help me spread the word about our new “Machine Learning and Environment Postdoctoral Fellowship” at CU Boulder by sharing opportunity with anyone you know that might be interested!
The ML and the Environment Postdoctoral Fellowship is a great fit for researchers who focus in Machine Learning (ML) and/or Artificial Intelligence (AI) with connection to the field of Environmental Data Science (EDS). The fellowship is hosted jointly by our CS Department (where I and Claire Monteleoni will serve as advisors for the postdoc) and the vibrant and interdisciplinary Environmental Data Science Innovation & Inclusion Lab (ESIIL) within CIRES at the University of Colorado, Boulder. More details including how to apply can be found at this link: https://jobs.colorado.edu/jobs/JobDetail/?jobId=59901.
We encourage applicants to apply by Dec 15 for full consideration! Prospective applicants should feel free to reach out to me (esther.rolf@colorado.edu) if they have questions. (obviously anyone on this slack can just ping me :))
Hi all - BlueSky is really taking off - if you are looking an alternative to X that feels a lot like vintage Twitter, come join!
I am putting together a Starter Pack for AI for Conservation. Starter Packs make it easier for folks to find people follow who post on specific topics of interest, and a good way for you to build a following of people interested in your work. If you'd like to be included, comment your handle here or send me a DM there (@blaircostelloe.bsky.social) and I'll add you!
Hello, my alias is @lostanlen.bsky.social
thank you! 😊
It sure is taking off, my alias is @ urbanzoochory.blsky.social
Thanks all - I'll share the starter pack in the next couple of days. In the meantime, let me know if you'd like to be added or if you know of other accounts in this space!
@andyvhuynh.blsky.social would love to be added! thank you so much!
Hi my alias : skylord999.bsky.social
Us (the Kitzes Lab) please! https://bsky.app/profile/kitzeslab.bsky.social
And my personal handle is @tessarhinehart.bsky.social :)
Hey @Blair Costelloe - great to see! Long time no talk. Please add me too (and I have a very draft-y one with NLP x climate/conservation peeps) - @harpactes.bsky.social. Hope all is well! ❤️
Just signed up, would love to be included too, thanks! @vdplasthijs.bsky.social
Hey, great idea. Add me in!
@eugenegalaxy.bsky.social
Thanks a ton! @christopher-wang.bsky.social
Does anyone know why their handles have to end with .bsky.social? Seems excessive...
It's basically part of the AT protocol, on which BlueSky is built. More info here: https://bsky.social/about/blog/4-28-2023-domain-handle-tutorial
i.e. you can change it to use your own domain name IIRC.
All added! I suggest you add a photo and brief profile description to your account, if you haven't already. This will make it more likely that folks will follow you.
Thanks for doing this! @bletcher.bsky.social
I would be honored to be in the pack: @johdolgha.bsky.social
@capreolus.bsky.social@capreolus.bsky.social
Can you add me too? @yteng.bsky.social
Thank you 🙂
Cross-posted the news to Wildlabs discussions. You can find it here:
https://wildlabs.net/discussion/ai-conservation-starter-pack-bluesky
Lets gather more people. 🎉
Thanks for posting @Eugene Galaxy!
We still have a ways to go before the pack is full, but once space gets tight I plan to do some curation and will prioritize accounts with profile text and pics that post at least semi-actively about AI + conservation. There's always the possibility of a second pack as well.
Thank you very much for doing this! @johnmartinsson.bsky.social
late to the party: kiedanski.bsky.social
Hi! Mine is @brookedgoodman.bsky.social . Thank you!
late to the party but @justin-kay.bsky.social 🙏
thank you for putting this together!
Small ad for a data science role for wildlife ecology in Fort Collins, CO. Closes 11/29/24: https://www.usajobs.gov/job/820372600
Sharing on behalf of Sabine Loos from the organizing committee: ACM COMPASS 2025, hosted at the University of Toronto, invites interdisciplinary research on sustainability with a focus on how computing and moves toward sustainability are embedded in and shaped by specific local contexts—computing in place. Rooted in the broader discourse of sustainability but also open to questioning its prevailing narratives, COMPASS welcomes work that critically examines how technology intersects with social, environmental, and economic systems. This year's theme encourages a deeper, situated perspective on how technological practices emerge as solutions, but also contributors to the complexities they claim to address.
COMPASS draws from a wide array of disciplines: computer and information sciences, social sciences, critical geography, environmental studies, economics, climate studies, and engineering. We aim to foster a discussion that interrogates not only the potentials but also the limitations of technology-led interventions in sustainability. This includes research that explores power dynamics, inequality, and extraction—how technology might reinforce or disrupt existing systems. We encourage work from activists, worker advocates, and indigenous scholars, alongside those studying topics such as labour movements, climate adaptation, and the ethics of sustainability practices. COMPASS values research that brings forth the often-unseen political and material consequences of digital infrastructure in shaping futures.
COMPASS explicitly promotes multi-disciplinary and cross-disciplinary research and practice that address key challenges for sustainable societies including, but not limited to, equity, health, education, poverty, accessibility, conservation, climate change, energy, infrastructure and economic growth. We also welcome research on the ethics of technology, especially from a critical perspective, that explores limitations and concerns with technology-led solutions for sustainable societies. COMPASS also has a particular commitment to approaches and research that address the challenges faced by under-represented and marginalized communities.
The theme of the conference will be Computing in Place. We also invite submissions that speak to that theme and/or contribute to one or more of the following areas:
- Systems and IoT for Sustainable Societies
- HCI, Design and Critical Perspectives
- AI, ML and Data Science for Sustainable Societies
- Development, Economics and Policy
- Environment, Sustainability and Climate Change
- Technology, Media, and Social Practice
Papers:
In our papers track, we welcome a variety of submission types, including deployment experiences, practitioner reports, user studies, ethnographies, design research, and systems building. We invite contributions from a range of fields, practices, and individuals who may not see themselves as traditionally members of 'computing' disciplines, including the humanities and social sciences, climate and sustainability research, and community advocacy. Accepted papers will be published as a conference proceedings with the ACM. Authors will have the opportunity to present their work as part of either in-person and hybrid paper sessions.
Important Dates for Papers:
- Submission site open: January 10, 2025
- Paper Deadline: January 17, 2025
- Notifications: March 21, 2025
- Camera-Ready Completion Deadline: April 7, 2025
Deadlines are anywhere on earth (AoE) time.
Please find full information about the call at: https://compass.acm.org/call-for-papers/
Late-Breaking Work:
Accepted Late-Breaking Work submissions will be included in the ACM Archives and presented during a special posters session at the COMPASS conference. Authors must therefore expand or otherwise significantly modify versions of these works as papers to COMPASS or any other venue. This year, COMPASS is accepting late-breaking work submissions in multiple rounds. Authors who need visas to travel to Canada may consider submitting to the early round.
Important Dates for Late-Breaking Work:
- First Round
- Submission Site Open: November 25, 2025
- Submission Deadline: December 1
- Notification to the Authors: January 15
- Camera-Ready Completion Deadline: February 7, 2025
- Second Round
- Submission deadline: February 28
- Notification to the authors: April 15
- Camera-Ready Completion Deadline: May 7, 2025
Deadlines are anywhere on earth (AoE) time.
Please find full information about the call at: https://compass.acm.org/call-for-late-breaking-work/
Anyone heading to WACV and potentially looking to share accommodation?
Room rates are apparently quite expensive and I'd be happy to room share
A small insight into one of the NeurIPS poster sessions: 15 minutes before the closing, it is still pretty packed.
Wow that’s really cool! Hopefully I’ll get to attend next year
NeurIPS 2024 Workshop on Tackling Climate Change with Machine Learning
Stream the whole workshop live & for free via the website:
https://www.climatechange.ai/events/neurips2024
sometimes zero-shot generalization works in unexpected ways. Deepforest bird detector apparently also gets manatees? In case anyone out there wants to start from that backbone.
Do people know of any DJs or (electronic) music that samples bird sounds into the music (bonus if it the birds are from New Zealand?).
I suddenly felt an urge for this, without having an artist base to look from. 🌟 🎵 Thank you.
also there’s this if you want to DIY: https://academy.allaboutbirds.org/beastbox/
Not birds, but maybe bird/sky adjacent? I met someone yesterday who has an ambient project that uses weather balloon data to make music. https://monsoonstation.com/
This is a very silly whale version that's fun to play with https://whalebeats.bluewhalesfilm.com/
"Happy Cycling" has a legendary seagull (I think?) sample: https://www.youtube.com/watch?v=MEFSU7WeZt0
The Phoenix Foundation (NZ indie band) often sample native birds in their songs (e.g. Supernatural)
this is super random and almost not related to your question. I'm interested in the creation process and came across this video of the making of J. Cole's "No Role Modelz" with Phonix Beats. He sampled bird vocals into the beat and used them as a hi-hat. I cannot unhear the birds in the song anymore haha. Starts at 5:34 https://youtu.be/Xd1wEy1j9No?si=L3uQhinLVpNFxHxW&t=334
https://www.youtube.com/watch?v=Sc45MX6naUU
Dominik Eulberg. One of the best! https://dominik-eulberg.de/apus-apus/
https://www.museumfuernaturkunde.berlin/en/wissenschaft/dominik-eulberg-ein-botschafter-fuer-natur
My husband uses lots of nature samples in his ambient music. There are bird sounds in there too, but I'm not certain which songs exactly: https://arrowounds.bandcamp.com/
Check out this playlist: https://open.spotify.com/playlist/38ABjTam1XalavZAhxZyFt?si=2cf4139e97f344c5 I love the Ela Minus track
We sampled farm birds and made a song out of it https://open.spotify.com/album/5ZbMblOYbOBz5Mt26VPmmL
Also, check out this lecture on integrating rules of collective behavior (e.g. flocking of pigeons) into music leveraging computer vision AI https://www.youtube.com/watch?v=nYN5RS6r2ZE&themeRefresh=1
I am very late and perhaps off-base, but I immediately thought of this. The first track does start with birdsong! In any case, it’s very organic, natural-y electronic/ambient. I affectionately refer to it as “whale music” 😂
https://open.spotify.com/album/0eW3b50zPiSOrK0SiG8ZEh?si=mHW3CVLzSpSG9ZO9WO_yfg
Thanks everyone for this epic collection and great suggestions of nature samples music.
I transferred the links into a document:
https://docs.google.com/document/d/1BQrsFH1PSb7LXTbYDThJe7pfbVyaFib-Jj8OhNd2SkI/edit?tab=t.0
🐸 https://lechugazafiro.bandcamp.com/album/desde-los-o-dos-de-un-sapo
The first day of the 1st Ecoacoustics & Bioacoustics Congress in Colombia was only about art. Alejandro Bernal composed this piece based on the sounds of 20 endemic birds of Colombia. In the description, you have the species and the original audios that were used for the composition 🫶https://www.youtube.com/watch?v=x3A9TGCr_8Y&ab_channel=InstitutoHumboldt
Forgot to mention this one in this thread.
Imogen Heap and Dan O'Neill did a "nature sounds" album.
While it's not specifically bird sounds, it's got plenty. One of the tracks is even named Kākāpō.
https://www.youtube.com/playlist?list=OLAK5uy_nkq6y4Waftxw2PR9i8inrc3KSPjuatApY
https://www.youtube.com/playlist?list=OLAK5uy_kY6iU2pt3d99_Ms2SVk3HCZbpV-2La0BA
Also I just came across Londonderry Air by Beatrice Harrison. 2 of my favorited things, cello and birds! And rumours that the bird was faked are unfounded apparently. https://www.theguardian.com/music/article/2024/may/22/the-nightingale-beatrice-harrison-radio-bbc-cello-duet
I've always really loved Takashi Kokubo's music - this album (https://takashikokubo.bandcamp.com/album/sound-of-the-earth-vol-9-sound-trekking-oceania) has some amazing field recordings from Australia and New Zealand, some specific to birds 🙂
Ah did no-one mention this one yet?! https://shikashika.bandcamp.com/album/a-guide-to-the-birdsong-of-western-africa
🪸🪸🪸 https://balmat.bandcamp.com/album/projections-of-a-coral-city
Sending a message, to keep the thread alive. it seems like it doesn't go away if there's a post every 30 days or so. Thanks everyone for your contributions, and keep them coming 😄
Hi, everyone! I’m working on a task to identify animals from footage taken from a helicopter. There can be as many as a hundred animals in a frame, and its very difficult to differentiate one animal from another of the species. I’m trying to find a way to re-id the animals when they enter and re-exit the frame. The tracking algorithm I’m currently using is BoT-SORT, but with the identifiability of each individual animal being so low, I suspect that algorithms like DeepSORT or FairMOT won’t be too much more effective. Does anyone know of anything that might be able to help with re-id in this case, maybe a tracking algorithm with a strong memory specializing in tracking objects when they go out of frame?
Are you able to actually achieve this as a human? Having extremely low resolution, fast moving imagery + fast moving objects, and many confusing objects is sort of the worst possible case recipe for this kind of stuff.
We had a thread on something similar awhile back (I'm afraid it might've been deleted by Slack?), but my guess is that this will rely pretty heavily on good localization of detections rather than relying heavily on appearance-based re-id. (really trying to line up global imagery and/or backing out helicopter motion and then lining up local patches and maybe local motion?) I'm not sure most off the shelf stuff will work super simply without a lot of work.
i haven't used, but the new segment anything 2 release claims some re-id component and i've wanted to play with it.
what kinds of animals are we talking about?
Thanks for the feedback! Do you have any recommendations on out off box tools for object localization that I could maybe modify for this task?
can spit out images, retrain our livestock detector, should be pretty easy on per image basis. The camera motion for double counting is 2nd level once thats done (harder).
I have some workflows for double counting but still just scripts and experimental -> https://github.com/weecology/DoubleCounting, mostly just inspiration, needs more work to make functional.
I don't know where @David Russell is on his latest work in this area.
Since it is contiguous, maybe you could tile the footage to generate one large composite image?
At Open Forest Observatory we've been developing a tool for projecting per-image predictions into geospatial coordinates using the output from photogrammetry tools such as metashape or COLMAP. Overall it's pretty mature, but the focus is mostly on veg cover segmentation applications. We do have a little bit of work on detection tasks (using Ben's data) in this notebook here. One approach is just to project each detection into the geospatial coordinates. The other casts a ray through the center of each detection and tries to triangulate multiple detections. In theory, the later approach could work well even with strong camera motion, but would assume the objects aren't moving much.
This is very prelim work but I'm happy to chat more about it if you're interested. We may be revisiting it a bit more in the context of tree detection some time later this year.
<a href="https://github.com/open-forest-observatory/geograypher">open-forest-observatory/geograypher</a>
both ultralytics and norfair have a re-id component in them that might be worth trying:
https://github.com/tryolabs/norfair
https://docs.ultralytics.com/
@adam dunlop Are you just looking for something to label the species in the footage (such as zebra or giraffe) or are you looking to identify specific individuals as well (such as Fred and Ethel the zebras)?
@Anastasia Pagán If we could identify Fred and Ethel individually that would be lovely, but I don’t think its feasible with the quality of data we have
@Diego Kiedanski Are you referring to the BoTSORT and ByteTrack trackers for Ultralytics? I’m having trouble finding the Re-ID components
@Jason Holmberg (Wild Me) @Jon Van Oast does this sound like a good candidate for Scout?
Sorry, norfair has re-id https://github.com/tryolabs/norfair/tree/master/demos/reid
Possible tool for stitching: https://docs.opencv.org/3.4/d8/d19/tutorial_stitcher.html
@adam dunlop I have worked on something slightly similar but with Fish underwater. I used post model re-identification methodes since most Trackers don't tackle these situations very well. You can try to fine tune bot sort or byte track and adjust it's parameters for beter tracking too. You can have a look at my method on my github. If you have questions feel free to message me. I am currently working on an improved method for aerial data.
https://github.com/ChaimElchik/Master-Thesis/tree/main/FinishedPythonFiles
What is your favourite social media these days? 👀
I would like to understand more about the wildlife community consensus (if there is such?) on social medias.
With recent BlueSky rise and X platform owner's instability, I am trying to understand where do you prefer to spend your time reading news, finding new content, and generally having fun?
Let's keep it within the nature & wildlife context, and not mention platforms used exclusively for private/family purposes (like Facebook for me).
Cheers! 🎉
Join Us as a Mentor / Tech Demo presenter for WECode 2025!!
Hello!
Hope you are doing well! I’m Hannah, a junior at Harvard, and a Co-Director of Engagement at Harvard Women Engineers Code (WECode).
Harvard WECode organizes the largest student-run conference for women in CS in the world. In past years, we have had over 1,000 students from 40+ different countries join us to hear from tech professionals at the forefront of their industries and learn about opportunities in college and their future careers. We are in the process of sourcing more mentors and tech demo presenters for the conference, and we’d love to have you so attendees have the opportunity to learn about tech careers oriented towards social impact. This year, our conference is scheduled for February 22-23, 2025, fully-in person on Harvard Campus and virtually on a platform called Gather. If you are interested in available, please fill out the following forms: In-person mentor interest form here and Interest for presenting a project either in person or virtually, here.
Thank you so much for your time, and feel free to send any questions my way.
All the best,
Hannah
Hello everyone! I'm trying to access images from the LIILA Hugging Face Colab, such as this image: https://lilablobssc.blob.core.windows.net/caltech-unzipped/cct_images/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
but I'm getting this permission denied error. Does anyone know if this is a general issue, or just me?
Is that associated with a human/vehicle label? All LILA-associated metadata - including the HF dataset - maintains links to human/vehicle images, but in most cases the actual images are not available in the public buckets. Caltech Camera Traps is one of the few datasets where a number of images do contain humans, but some have been removed. If you find the issue to be broader than that, let me know.
Also FWIW the HF dataset is woefully out of date and may or may not continue to be updated; all things being equal, I would consider working from the big .csv linked from this page:
https://lila.science/taxonomy-mapping-for-camera-trap-data-sets/
But if someone wants to update the HF dataset, that would be even better.
The images I'm trying to access are from the example cat images in the tutorial Colab. However, if they're unavailable then I can try the csv. Thanks!
By the way, are the image urls in the csv the same as the ones from Hugging Face?
Generally yes, although the .csv includes also includes the corresponding AWS+GCP links for every image. I'll take a look later.
Let me know if there is anything i can do
Ooooh, I see what happened. All the URLs in the Hugging Face dataset are obsolete, there's nothing unique about this image. So, that's the bad news.
The good news is that if you like that Colab and the HF dataset, and you don't mind the fact that it's a couple datasets out of date, you can add one line of code to the Colab to update the URLs. Only the prefix has changed, everything after "lilablobssc.blob.core.windows.net" is fine.
For example, this is the URL that doesn't work:
https://lilablobssc.blob.core.windows.net/caltech-unzipped/cct_images/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
These are the new GCP/Azure/AWS versions of the same image:
https://storage.googleapis.com/public-datasets-lila/caltech-unzipped/cct_images/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
https://lilawildlife.blob.core.windows.net/lila-wildlife/caltech-unzipped/cct_images/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
http://us-west-2.opendata.source.coop.s3.amazonaws.com/agentmorris/lila-wildlife/caltech-unzipped/c[…]ages/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
All of them end with:
caltech-unzipped/cct_images/59b93afb-23d2-11e8-a6a3-ec086b02610b.jpg
Hope that helps!
I'm right on the fence re: whether the HF dataset is doing more harm than good in its current state. I can't quite summon the enthusiasm to ask for it to be deleted, but also don't quite want to ask permission to take it over so I can update it myself, because then I have to do it. 🙂
If someone else wants to take over that dataset, it might be a good opportunity to learn about HF and contribute to the community, and I could ask Sasha (who did great work to get the dataset rolling) whether she would be willing to transfer it to a new HF organization.
That was really helpful, thanks so much again!
Hi Folks,
I recently picked back up a project I was working on last year, and wanted to share it with y’all. I’m calling it Prospect, it’s tool that allows you to instantly make your own few-shot classifiers on aerial/drone/satellite data. Just draw polygons to capture a few examples and you can see feedback almost immediately. I made a little video showing how it works here: https://www.youtube.com/watch?v=p15OLgjwmKI
If you wanna give it a shot you can trial it on a few pre-made scenes here, sign up with invite code SHINE. Once you have an account, try making a new task in any of the 3 scenes: https://prospect.moonshinelabs.ai/
The current app is pretty raw — I’m about as close to MVP as you can get (and maybe not even quite there). I’d love feedback and to hear what directions might be useful for the stuff you all work on. For example, stuff I’ve thought about:
- Provide a map tiles link to let you import the map into your GIS tool
- Export to GeoTIFF
- Automatically create polygons or instance masks
- Multiclass
I don’t have self-service uploads of new scenes yet, but if you wanna try it out on your own data just provide me a GeoTIFF and I can upload it for you to try! Also this thing is currently held together by duct tape, so apologies in advance if it’s buggy today, especially with multiple people using it at once
Great work and good luck on the finishing touches! I work for a forest accelerator and will be sharing this with our GIS team.
What are your goals for the project? What license might it operate under? Is it hosted on GitHub by chance? Really great work!
I am unable to sign up and try this. Or are you reaching the server limits?
🌊 ATTENTION ALL OCEAN ENTHUSIASTS AND AI FOR CONSERVATION FOLKS LOOKING FOR A NEW HYPERFIXATION!
FathomVerse mobile game debuts new features to help gamers participate in ocean exploration
FathomVerse is a free mobile game that invites everyone with a smartphone or tablet to contribute to science. By playing blissfully productive mini-games and interacting with stunning ocean imagery, gamers partner with researchers to identify ocean animals and train AI used for ocean exploration. FathomVerse is integrated with the FathomNet database and aims to expand the library of annotated ocean imagery.
The FathomVerse team improved the game to enhance player engagement, foster community connections, and expand scientific contributions. Play FathomVerse to help scientists discover all life in the ocean!
Learn more: https://www.fathomverse.game/blog
https://youtube.com/shorts/R0xjc4LW0zs?feature=share
I have resigned as a reviewer for the Royal Society. I encourage others to do the same. My letter is here.
I managed to get the LILA dataset link included into the official “how to do the final projects” deck presentation 🥳 🥳 🥳
I work with Le Wagon sometimes as a Teacher of their Data and ML bootcamp. They do final projects over two or four weeks where a group of students picks a dataset and then they work on it - there are multiple batches all around the world who will now see that this kind of data exists too and maybe get inspired to look & contribute to this space.
Thanks @Dan Morris and everyone who is making it accessible.
I know it’s just a link somewhere, but little by little, a lot 💚
🌱
Hi everyone! Hope you're hanging in there. Considering everything that's going on in the US, and how many of our colleagues from the federal service there are recently out of work, I was wondering whether others would be interested in having a "jobs" channel? I know as a postdoc it would be great to have another resource for listings in the conservation space, and thought others might feel similarly. However, it's obviously not my call. Thoughts?
oops! I don't know why it hadn't shown up for me! Thanks, Sara :)
Just wanted to share a short film about hardware engineering for conservation bioacoustics 🙂
https://www.youtube.com/watch?v=9ieQ9Qlpt2s
🌲🦉 🐘 🐋 🌳
Feels appropriate here: https://arstechnica.com/science/2025/04/google-created-a-new-ai-model-for-talking-to-dolphins/
for my colombia workshop, i made a simple video for installing label studio and getting started, for anyone who wanted to see https://www.loom.com/share/fdc99e706b6741ad8632c78ca3c07b9f?sid=f5f4ba7a-a972-4eb4-a86a-e310408c028d
A random project I did for a class where I tried converting insect images into 3D models! The results were not as good but I was highly suggested to make this for d&d too :) https://devmehrotra.me/insectclassifiers/
Hi everyone! I’m an environmental writer currently at MIT. I’ve really enjoyed following this community and I’d love to help expand its impact through storytelling. It’d be great to hear if anyone is seeking support in science communication/content creation or knows anyone who does!
Sorry for the spam, but UK folks: https://act.for-the-ocean.org/act/protect-uk-marine-havens
Hi all! I hope this is an okay venue for this question: I am looking to get a new laptop for work and wondered what computing systems others in this space are working with. My current computer is a Macbook Pro (2017) with 8 GB RAM and 512 GB storage-- I'd like something with at least as much storage and am hoping to up the RAM. I'm open to Macs/PCs; I've worked with PC desktops but have never had a PC laptop. I generally use my computer for large scale audio data processing and analysis, including BirdNET in CLI, Python, QGIS, and R. My Macbook has generally been okay for this, but it is absolutely on its last legs. I've heard good things about the M2-M4 chips. Any advice on choosing a new computer as an AI in conservation researcher would be appreciated! Thanks!!!
This is a bit of an aside, but our group has really benefited from using free NSF-funded cloud computing through Jetstream2. It provides a nice in-browser web desktop so you can even run GUI applications. And one of the nice things is you can change the size of your virtual computer on the fly so you can prototype on a small one and then run your final experiments on a larger one. If you're a US-based academic researcher you can just fill out a simple application and my understanding is they are almost always approved.
thanks David! I am currently based in Canada. I need a new computer regardless, but will also look into this :)
Hi everyone! I'm in Boston presenting a workshop at ODSC. I'll be here until this Saturday (May 17). It's my first time in Boston and I'd love to visit MIT while here. If you're at MIT and open to meeting up or chatting, feel free to reach out 🙂
Hi Hi!
I'm excited to have just joined the group and really looking forward to learning from all of you. My background is in AI and Robotics, but I’ve always had a strong passion for wildlife conservation. Lately, I’ve been exploring how edge computing could support conservation efforts, and I’m particularly interested in starting with camera trap applications. Before diving in, I wanted to ask if there are any existing projects or models that are already optimized for this kind of work.
Thanks so much, and I’m looking forward to connecting!
https://blogs.microsoft.com/on-the-issues/2024/12/18/announcing-sparrow-a-breakthrough[…]-and-protect-earths-biodiversity-in-the-most-remote-places/ @Zhongqi Miao and pytorch wildlife team.
Our first batch of sparrow units are already up in the Amazon rainforest!
WOW! That's HUGE. Is it open sourced already ? Would be curious to see how it works . Thanks for sharing @Ben Weinstein
We are in the middle of open sourcing everything. Please keep posted! The goal is to open source the whole software and hardware solutions so everyone can build it themselves
If you want to start early, let me know, I can send you the BOM and you can start building if you want to
Hello! My name is Joban, and I am an undergrad working at UW. I was wondering if there was a way to get the GPS info of the LILA API datasets?
Not sure what you mean by LILA API, but AFAIK the only LILA dataset that has publicly available GPS locations is iWildCam
Timm is correct, precise location information is removed from all the wildlife datasets on LILA by design, as are images of humans. This mitigates a lot of the risk of misuse and makes it easier to keep LILA available.
There are a couple of non-wildlife-related datasets on LILA with location information, e.g.:
https://lila.science/datasets/adirondack-research-invasive-species-mapping
If you just want wildlife occurrence data with location information, check out GBIF. If you want images with location information and you're not looking for a specific ecosystem (i.e., you can cast a wide net), I would search for the relatively small number of camera trap datasets with images on GBIF, and - depending on how many images you need - I would poke around the "explore" page at wildlifeinsights.org. Neither of those will get you to the volumes of images you can get from LILA, but if you want a small sample of paired location/image data, both are good approaches, as is the iWildCam dataset that Timm referred to.
Hello! By the "LILA API datasets", I mean the datasets that are collected on the website from whom we can download the images, megadetector results, csvs, etc. I guess I didnt use the right term, sorry!
Besides that, this is great. Do we know if datasets without specific country information, such as WCS, but have data from over a dozen countries have a way to map at least to country resolution?
And if not, would they say performing a match between species name in GBIF and country of occurrence is an adequate workaround for such dataset?
WCS Camera Traps has a "country_code" field for each image in the metadata file.
If you just want a series of occurrences with locations (e.g. from GBIF) and images of the same species, I would recommend not only ignoring any country/location information in LILA, but ignoring the divisions between datasets entirely. Instead, I would just use the big .csv file on this page:
https://lila.science/taxonomy-mapping-for-camera-trap-data-sets/
...which treats all camera trap datasets on LILA as a single dataset, and grab images of whatever species you're interested in. E.g., if you want locations of jaguars and images of jaguars that aren't literally from the same observations, it's extremely unlikely that it matters whether those jaguars came from the same country as those locations (there are scenarios where this could matter, but they are pretty esoteric, and I can't think of any images on LILA where this would matter).
Hi everyone!
My team and I are working on various projects at the intersection of machine learning and ecological research. Since science communication on these topics often falls short, we decided to create a small web series about our own work and that of others.
I’d be happy if some of you checked out our mini documentary on the topic of self-medication in bats! 🙂
👇 the mini documentary 👇
https://www.youtube.com/watch?v=tVv15MxI4Zs
☝️ thank you for your support ☝️
hi, some guy named Charlie seems to have hacked @Chloe ‘s account and is sending spam messages to inactive members
For fun I am going to say "yes" interested
I asked Sara in DM to remove this bot from the slack.
At the same time I recommend everyone to report the user directly to slack.
I will remove the user, this space is not intended to be used for advertising spam
A sad article in the Guardian about insect and bird decline 😪
https://www.theguardian.com/environment/2025/jun/03/climate-species-collapse-ecology-insects-nature-reserves-aoe
Read silent earth on the topic last year and it’s beautiful(ly heartbreaking)
Hi all 👋
Quick question . does this detection look like an Egyptian mongoose (Herpestes ichneumon) to you? Also, could you please confirm if the attached distribution map (source wikipedia) for the species looks accurate? Appreciate your input 🤗
thanks!
Posting for a colleague
Summer School
========================
WildDrone (which a number of Bristol colleagues are involved with) are holding a summer school aimed at PGs/RAs, at Max Planck Institute in Konstanz, 22-26 Sept 2025. 95 EUR registration, including conference fees, T-shirt + coffee/snacks. Lots of great speakers, including some you may recognise. Registration deadline 15th July.
The aim of the WildDrone summer school is to bring together multiple disciplinary approaches to wildlife conservation practice. We want to equip scientists with a unique combination of multidisciplinary skills for drone-based ecology and wildlife conservation. The WildDrone summer school is a one-week intensive training course focusing on questions in conservation ecology while incorporating novel research in drones and computer vision technologies. This course will be a forum for presentation and discussion of theoretical and practical issues of drones for conservation fieldwork applications.
https://wilddrone.eu/summer-school/
Best wishes,
Steve
--
Dr Steve Bullock
Associate Professor of Aerospace Engineering
Bristol Flight Lab, Faculty of Science and Engineering
University of Bristol
Adding to the thread @Dan Morris started on code automation. I met with a curator at Cornell on trying to increase access to bird recordings in the Macaulay Library and we identified one impediment is the laborious process of gathering catalog numbers. Based on our meeting I developed a prompt for a Cursor background agent "I am an open source python developer looking to create a well documented project on github. I want to create a tool that automates the instructions here:
https://support.ebird.org/en/support/solutions/articles/48001064551-using-and-requesting-media
The input to this tool takes in a species list, looks up the species in the ebird taxonomy from
https://www.birds.cornell.edu/clementschecklist/introduction/updateindex/october-2024/2024-citation-checklist-downloads/?__hstc=60209138.79cd88ffe39899823b6ce6abf82368c6.1742826523426.1749937345635.1750024212234.9&__hssc=60209138.1.1750024212234&__hsfp=287177776&_gl=1ik4qdi_gcl_auMTg2OTc5OTMxNS4xNzQyODI2NTIy_gaMTcyMTQzODg4My4xNzQyODI2NTIy_ga_QR4NVXZ8BM**czE3NTAwMjQwNjAkbzE0JGcxJHQxNzUwMDI0MjEyJGo2MCRsMCRoMA..
Then finds those records in the Macaulay library. For example if I wanted
“An american robin in New York in May” I would format this url.
https://media.ebird.org/catalog?view=list&all=true&taxonCode=amerob&mediaType=audio&tag=song®ionCode=US-NY&beginMonth=5&endMonth=5
I then want to extract the catalog IDs, they will look like
637255739 for this page https://macaulaylibrary.org/asset/637255739,
Additionally, I would like to be able to look up a species list from the ebird API instead of inputting a species list.
https://documenter.getpostman.com/view/664302/S1ENwy59
Be creative in creating a tool that users will find useful and easy to understand.
The output for the user is a csv file with the metadata used to generate the query (e.g. geography, species, date) and the catalog ID number.
Add tests, github workflow instructions and other tools to help ensure reproducibility.
Push everything to
https://github.com/weecology/MacaulayLibraryLookup.git"
It costs about 4$ in API calls (i've already subscribed to cursor monthly fee). It did 99% of the work, but needed about an hour of my time to clean it up. I can't really estimate how long it would have taken me, I wouldn't have attempted it. Definitely amazing, it achieved almost all goals -> https://github.com/weecology/MacaulayLibraryLookup. I feel weird about representing this as my work given how little I did. (https://github.com/weecology/MacaulayLibraryLookup/commits/master/).
really amazing - thanks for documenting this for us. really interesting possibilities here, it seems. i wonder if this general topic of, um, assisted development(?) merits its own channel in this slack?
I'm open to whatever people want, I thought about just putting it in general. In terms of its capacity I think having examples is really helpful, one failure point was it couldn't download the taxonomy.csv, I had to go do that, and it did alot of dancing around that instead of just stopping and asking me to help.
fwiw @Ben Weinstein, I tried out the tool and found it can only get up to 30 records per query, presumably because that’s how many the web page loads until you click ‘load more’. Cool that you built this with agentic AI! I think this limitation is a good example of how you can build an MVP very fast, and it will make some tests that pass, but it might not behave exactly as you imagined! (Also, I bet pushing cursor here, it might find a workaround that does retrieve all records, but that might end up being substantially more difficult at least taking the current approach of parsing the html). I wasn’t sure if I should open an issue on the github, eg if this repo is meant as a playful demo or as a real tool
(the best solution for my actual use case was to follow the instructions of doing the html query then clicking the ‘download csv’ button)
No, I think we should be able to handle that. I won't have a second today, but I wouldn't give up so fast. To push back, I think developers limit their ability to imagine AI tools to be able to do this stuff and relegate it to playful demo. From what i've seen, you can push it further. If you can page through the results, so can it.
As a test of the concept, make an issue, and I'll assign it to co-pilot, I want to see if it can handle it by itself.
I started a youtube channel on "5 minute papers on AI for the Planet": https://www.youtube.com/@AIforthePlanet (let me know if there are good papers I should consider covering)
Love this!! Great videos! I love how approachable you present the material
Maybe @Björn Lütjens paper would be a good fit for this series (even though you already have one regarding satellite images) https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10758300
Are there any attribution science and conservation people here? Would love to hear about projects!
This is hilarious and I suspect folks here might enjoy - cuts to the chase at the 17:00 mark
https://youtu.be/hCQCP-5g5bo?si=e6ChGuyMw4NKhkz0
It's fantastic that "I saved a PNG image to a bird" is not a deep metaphor, he means it literally.
I thought it was clickbait for sure but he does indeed mean literally - I'm still shook
I often wonder whether certain animal species are painting images for each other using sound- if a dolphin can echolocate using sonar and ‘see’ the reflected sound bouncing off fish, can it also deliberately produce images using sound for other dolphins to see? Can birds do that?
I am inspired and have been rickrolling nature through audio playback all afternoon:
Hello everyone! 🐟 🌱 🦌
Here is the first episode (of hopefully many) on the topic of Tech for nature:
https://open.spotify.com/episode/39nW4dBnuH8vAkk8WnjEcs
As I met people from "this" community, I always felt so privileged to be on the receiving end of the inspiring, empowering and meaningful stories, and felt like more people should hear them. So here is @Peter van Lunteren's.
Happy to hear any thoughts, feedback, ideas. Thanks (also for doing what you are doing)
Hey everyone. What conservation conferences/symposiums/events are taking place in Europe this year that you know of?
I know about these 3, do I miss something cool?
https://networknature.eu/networknature-annual-event-2025-choose-nature
Brussels, 16 September.
https://rewilding.academy/rewilding-our-world-conference-2025/
Netherlands, 20-21 September.
https://www.nordicsocietyoikos.org/news/nordic-oikos-2026
Odense, 14-17 September
Anyone is planning to attend any of these ^ 3?
Edit: Nordic Oikos is 2026. I somehow thought we are already here...
Thanks a lot for the list. Do you have experience with the Rewilding conference? I am thinking to go there.
I haven't been there before, this will be first!
There is TIBS if you are doing biogeography: https://conferences.au.dk/tibs-aarhus-2026
Right under my nose. I am from Aalborg, that's a 1.5h drive from Aarhus.
it's not conservation only, but there's this event in Switzerland on Oct. 1st : https://ai-environment-summit.com/
(Sara Beery is one of the speakers)
The International Bioacoustics Conference is coming up soon in Denmark!
Cool Wildlife Insights feature!! https://time.com/7312183/general-catalyst-ai-apps/
Is somebody currently at the International Wildlife Conference in Lillehammer? 🙂 If so would be happy to meet you. 😉 I will be also presenting during today‘s poster session some of our drone based deer monitoring work 😄
Similar question to @Christoph Praschl's, is someone going to NetworkNature Annual Event in Brussels, next week? Would be happy to meet you and have a chat!
Also if you are located in Brussels/surroundings but do not attend the event -> we can still do that! 😁
I am in brussels a few times of the year, would be great to catch up. (but not for the conference)
Hey everyone! 👋
I'm Gerard, from Barcelona (Spain). I just joined this group, and I'm genuinely inspired by the fantastic work you're all doing. I'm eager to gain some real-world perspective from people working in this space.
I have a Bachelor of Science in Artificial Intelligence and have spent two years conducting research in machine learning. I am now considering pursuing a PhD, but I'm uncertain about which direction to take. I'm exploring completely different fields for my doctoral studies (I'm mostly interested in AI for the Planet, and AI for Neuroscience). The neuroscience path seems to have more traditional research opportunities (publishing in top-tier journals, etc.). Still, I keep getting drawn back to conservation work because I could have a greater impact in the world.
I've been lurking here for a bit, and the problems you all work on seem both complicated and vital. But I'm trying to figure out if this is actually a good path for someone like me, or if I'm having "save the world" feelings that will fade when I hit the reality of PhD research 😅
I'm really curious about a few things:
- Are there significant unsolved problems in this field that someone with a strong AI background could make substantial progress on, not just incremental improvements, but real breakthroughs?
- For people who did PhDs in this area - do you feel like you're making actual conservation impact, or mostly just writing papers?
- Is this field mature enough to offer good career paths, or is everyone still figuring it out as they go?
- What are the problems that look solvable from the outside but turn out to be way more complicated than expected?
I'm interested in a wide range of topics, from remote sensing to wildlife behavior to developing new monitoring tech. However, I'm also mindful of not wanting to invest 5+ years in something that sounds cool but doesn't actually have a significant impact.
Anyone willing to share their honest thoughts on doing research in this space? The good, bad, and "wish someone had told me this before I started" parts?
Thanks for reading this long post! I'm eager to learn from your experiences and insights, and would be happy to chat more about anything that resonates 🌍
I have also been lurking because I want to submit some applications this cycle and was thinking about making a post similar to this 😅 very interested to see peoples replies
Hi Gerard, I feel like I am almost in the exact same boat as you 😂 I've always been interested in nature and I did my bachelors in CS and conservation studies, but now I am doing a postbac RAship in neuroscience, and I am considering what I should do for my next cycle of phd application. I'd also like to see what people think for the questions you asked! On the more practical side, I'd also like to ask, is there a good way to combine these? For reference, my current postbac experience mostly focused on designing stimulus and processing data from human fMR experimentsI , and I feel like this is completely unrelated to AI for planet. There are some skillsets that are probably transferrable but i can't think of too much at this point. anyway, I am currently in this state where I'm not sure which area I want to pursue if i were to do a PhD. If anyone has been in a similar position I'd love to chat 🙂
@Gerard @Eric Chagnon @Yuerou Tang Conservation definitely needs folks with your skills! I recommend a personal "gut check":
- Do you want to help with the framing around how AI can influence better practices in conservation, or do you want to develop new AI techniques that can generally help a broad category of problems? The PhD route might be for you.
- Do you want the rush and sleepless nights of trying to solve a real-world conservation challenge with AI? Then I would recommend partnering with a field organization doing on-site work on a species of interest to you personally. This is somewhat of the route I took.
The good news? They're not mutually exclusive! You can do both! Just realize that both are different ways of thinking about AI for Conservation. Both are helpful. Both have different sets of expectations around what "impact" is, and you should choose a path that is meaningful to you and matches your personality.
My tip:
You need much more information to take such a decision.
Try to taste some of each world by participating in hackathons, open source projects, developing a simple idea you had in mind, etc.
There is another path which is not mutually exclusive either - being a freelancer. You won't get a fancy degree, you probably find yourself in the "thanks" of a publication, but you can live pretty well, travel around the world while doing so (as digital nomad) or stay wherever you'd like. Once you had enough or met the project of your dream or decided to learn something else, switching is easy and sometimes even not required to switch.
What I'm trying to say - ask yourself what are your priorities in the near future (~5 years) and which path will answer those priorities and interests.
Who you'd expect to be within 5 years?
Bare in mind one important thing - the Conservation title is the highest value within all the projects around here, but... as any other field driven by values (and almost no immediate results), most of it is the everyday work.
In my opinion, if you hate the daily effort, no value and no title will pull you out of bed.
If you love it, the values are a great clean and green fuel.
Are you doing the work you described here/ a freelancer? Curious to hear more.
Yes.
I'm part time on salary and mostly freelancer.
https://alignmentalignment.ai/
Thanks so much for all the thoughtful replies 🙏 — it’s been really valuable to hear both the upsides and the challenges. And it’s encouraging to know I’m not the only one navigating these choices! I’m leaning toward a PhD path that combines foundational AI research with real-world deployment in conservation, so I can contribute both theoretically and practically. It’s also motivating to see how many top-tier papers are already out there in this space (and how much more room there is to grow).
If anyone here knows of research groups working at this intersection — strong on the theory side and committed to field impact — that are looking for PhD students, I’d be very eager to hear!
IBAT core datasets are now available on Cecil for commercial use
We have a number of announcements this week but this is probably the biggest - the IBAT core datasets are now available on Cecil for commercial use.
These datasets have historically been challenging to access for commercial use but they're also becoming requirements for various compliance frameworks. If you're interested in discussing please let me know! 🙏
What do you think about Nature-Credits?
European Commission released a roadmap this July, outlining their motivation and plans. Nature-credits, similar to carbon-credits, should reward (monetize) wildlife organizations efforts which are "currently go unrewarded" (quote of an EU parliamentarian).
I have recently attended a NetworkNature EU conference in Brussels where nature-credits was one of the hot topics. The crowd was more of a "suits" than "boots on the ground". There I've learned there are 2 camps of opinions - lovers and haters. The former support the idea of giving wildlife organisations an extra revenue stream and transparency (credits require strict audits), while the later dislike the idea of yet another convoluted financial-scheme that can go astray and become a brandwashing tool.
What is your take on it? I do realize majority of the readers are from USA and might be unaffected, but still.
P.S. I've spoken to a EU Commission member who is in the "nature-credits" team and he commented, that for the next 2 years the Commission's plan is to setup expert/test groups, and investigate to which extend the idea should be materialized - on voluntarily basis (philantrophy) or regulatory (like carbon-credits with industry required to keep their CO2 emissions in check).
Not my area of expertise but very interested to see how this plays out. Whether or not the "boots on the ground" accept it as appropriate or even achievable, those in "suits" will demand it. Looking to carbon credits, on the one hand carbon is far easier to quantify than the abstract concept of 'biodiversity', and yet carbon credits have still seen many failures. But, perhaps it gives a model that we can learn from to help Nature-credits get off to a better start. I imagine we'll see the same cycle of: hype, over promising, failings, picking up the pieces to do it a bit better, repeat.
AFAICS here in Australia, like carbon-credits or -offsets, biodiversity credits (or offsets) are too often used to enable further ecological destruction - but providing some financial support to some programs elsewhere. Surely we have some ecological economists or others here with more direct experience? Perhaps some who have received support from these schemes (which I know exist here in Australia) could comment. For ref, see these links:
https://www.environment.nsw.gov.au/topics/animals-and-plants/biodiversity-offsets-scheme
https://www.dcceew.gov.au/environment/epbc/approvals/offsets
and for critique:
https://www.acf.org.au/news/set-and-forget-offsets-are-killing-biodiversity
Hi, I don't know if this is the right channel but is there anyone here plaaning to attend ICCV CV4E Workshop on October 20th?
I really enjoyed this technical overview of land conservation in China, if anyone else is interested in the topic.
https://www.nature.org/media/china/chinabook-execsummary-highres.pdf
Back in 2023 I did the 2nd (3rd?) CV4Eco workshop and could not recommend it highly enough. In case of interest to others looking for courses, I wanted share that earlier this year I did another intensive 6-week course was at the Machine Learning Institute in London.
Over the course of my PhD I've become highly specialised in what feels like one niche topic of AI (CV for coral reef audio). I'd never really had the chance to play with text, NLP, multimodal models etc. The course was a brilliant opportunity to take a deep dive in all those other area's of AI I'd not yet had an excuse to dive into. It massively improved my ability and confidence to start using the latest research in area of AI, starting with models like word2vec, or coding our own transformers, then hacking models like whisper, QWEN, and even training our own voice control interface for our laptops then demo-ing this. In a similar vein to CV4Eco, its run in a workshop/hackathon type style. Some details from my cheesy linked in post below:
Each week we worked in teams, following a rapid cycle of learn, build, deploy. Highlights included:
📰 Deep dives on seminal papers in machine learning
📖 My first foray into natural language processing with machine learning
🧠 Hand-coding transformers (the algorithm behind ChatGPT!) from scratch
🖼️🔊✍️ Going multimodal with image, audio, and text data
It's currently free to anyone with British Citizenship, though others did in fact travel from abroad to complete it. I gain nothing from sharing this, just wanted to stick it here in case its ever useful to anyone else!
How much is it if not a British Citizen? It says to "apply now" to find out more, but looks like they gather info before telling you the cost.
I'm not sure I'm afraid, but if you email Izaak (izaak@ml.institute) he can probably answer. I think there funding model is changing currently so it might be that they are figuring this out!
I imagine there are some birders on here so I thought I'd recommend this YouTube documentary that was released recently. Two brothers set out on a Big Year with little-to-no birding experience and put together an interesting and funny film. https://www.youtube.com/watch?v=zl-wAqplQAo&t=4s
Hi :)
Does anyone have experience optimizing object detection inference pipelines on Single GPU (on Windows)?
I am running inference on ~100k aerial images of African Savanna of shape 3800800 and I am struggling to keep my A5000 GPU constantly busy. The detection pipeline includes a YOLO object detector and a bounding box Classifier (Dinov2 + MLP). In case it is useful, you can have a look at the repo https://github.com/FadelMamar/wildetect.
What I tried so far:
- Multithreading: failing to solve race conditions between dataloading thread and inference thread
- Multiprocessing (new to it, so I don't know much...)
- Local deployment using Litserve + Async requests (too slow)
I'm happy to go through it in a call if needed 😁
Have you tried running yolo first with a high batch rate and then running DINOv2 with also a high batch rate, and if you are doing augmentations, maybe doing them beforehand and saving them so that the dataloading functions don't become a bottleneck? Let me know if this helps or if there are any more clarifications! Pytorch also has the option to add a mutliple thread function to the dataloader itself.
I can probably help with this!
A few things off the top of my head:
- What does your workload look like? Is this a batch processing thing or a streaming/realtime thing?
- Is this a one time job? How worthwhile is investing in the serving infrastructure?
@DEV MEHROTRA Thanks for your answer. I have been using a batch size of 32 so far. I am currently just extracting tiles/patches from the larger base images. Base images are kept in memory until they are exhausted (i.e. all tiles extracted).
How could the multiple thread function in dataloader be used in my setting for more optimizations?
@Nate Harada thanks for your answer!
- It is batch processing
- We plan to run it roughly a few times a week. It's an active research project
- we have limited budget and can't invest in the serving infrastructure. Just one big workstation with lots of RAM
actually you wanna just hop on a phone call later today? I’ll be around after lunch PDT
sure is 2pm PDT okay? I will send an invite to your email
The UNDP is developing the Nature Relationship Index (intended to be a nature-focused analogue to the Human Development Index, building off this recent Nature perspective). They're trying to get expert feedback in this short form — consider adding your input to this upcoming metric! 🌏
Please join The Global Expert Consultation to Develop the Nature Relationship Index (NRI) for the upcoming Human Development Report (HDR).
In under 15 minutes, you can make a real difference - and decide to share your advice directly with us and be acknowledged in the HDR (or remain anonymous).
Open through Oct 15: https://anu.au1.qualtrics.com/jfe/form/SV_eYi9gBoQyO0BvsG?sce=s27
I am afraid we are going to see more and more of these... this wolf vs puma (https://www.facebook.com/AJAnimalking/photos/trail-cam-captures-a-wolf-protecting-piglets-a-farmer-in-montana-set-up-a-trail-/1179921240908019/) got +3k interactions (https://archive.ph/XvAzr)...
Ariana Grande talked about how much she loves the Merlin Bird ID app during her interview on Amy Poehler’s podcast this week 🐦🐦⬛ I thought it was such a cool shoutout! I figure there have to be some folks from the Merlin/Cornell team in this group, so I needed to make sure you all knew that Ariana Grande is one of your users! 💖💚
As long as you are in North America or Europe 🙂 E.g., Colombia: it recognizes about 60%, Singapore 40% But yes, great when you can use it!
Hello :)
I am building DeepFix an intelligent debugging system for machine learning models.
I'd love to get feebacks from people frustrated by current ML debugging practices! 😁
Wanna to try it out on your data? DM me!
Hi all, Thank you very much Sara for inviting me to the Slack community. I truly appreciate it. I’m really excited about the AI for Conservation. I recently watched your TED talk, and that’s how I first learned about this initiative. It was incredibly inspiring to see how large-scale conservation challenges can be addressed using iNaturalist images and AI.
As an early-career researcher, I recently completed a postdoctoral fellowship at Georgia Tech, I work on animal behavior and biomechanics, and I’m very interested in contributing and learning from this community.
Hi Sunny! I'm currently at Georgia Tech as well. Let's connect :)
Dear all,
Come to Prague, Czechia in May 2026 to present your work at ESP Europe Conference at the session: Artificial Intelligence for Ecosystem Services: opportunities, challenges, and governance!! The ESP conference is becoming quite popular place for exchanging ideas across stakeholders regarding ecosystem services.
The submission deadline is 31.December 🙂 Any queries welcome 😉
Hello! We are researchers at Brown University, Northeastern University, and FAIR at Meta studying topics like the use of synthetic data, representation of long-tail data, and their social consequences. We have a 20-minute survey for colleagues to share their professional opinions on these topics. We invite you to take this anonymous survey if interested.
Data Handling Information
- Responses will be retained and analyzed only by Brown and Northeastern researchers
- Closed-ended answers will be reported in aggregate
- Open-ended answers may be quoted without attribution or identifying details
- The study does not constitute human subjects research under U.S. federal regulations (45 CFR 46).
The survey is linked below:
https://brown.co1.qualtrics.com/jfe/form/SV_0BqPpttNJ9QtRIi
We would also appreciate it if you could share it with colleagues who may be interested.
To contact the research team for any feedback or questions, please reach out to Jessica Forde at:
jessica_forde@brown.edu. Thank you again for your help!
Best regards,
Research Team
Jessica Forde, Chantal Shaib, Candace Ross, Arjun Subramonian, Adina Williams, Levent Sagun
Brown University, Northeastern University, and FAIR at Meta
MIT IAP 2026 course: Sustainable AI
MIT’s introductory program on sustainable AI! Explore the climate harms of AI, the potential climate benefits of AI, the net climate impact of AI, and strategies for sustainable AI. This is a free, hybrid, non-credit activity that is open to all.
Learn more and apply:
https://forms.gle/nam1oroPHHSX1nz17
hi all 👋 i’m in the process of formally setting up an independent research lab in india called qinārā ai, and was hoping to ask for guidance on best practices around formation and setting up an independent ethics review / advisory process. if anyone has experience with this, or can point me to useful resources/people who've done this before, i’d really appreciate it. thanks!
a bit more about me: aaravpillai.github.io
Special Issue: VGI and Remote Sensing for Sustainable and Climate-Resilient Environments
The deadline for our special issue “VGI and Remote Sensing for Sustainable and Climate-Resilient Environments” has been extended to May 2026.
We invite contributions that explore how Volunteered Geographic Information and remote sensing can advance sustainability, animal conservation, climate adaptation, and resilience across scales.
You are also very welcome to first send us an abstract to evaluate the suitability.
We look forward to receiving your papers!
More information and submission: https://onlinelibrary.wiley.com/page/journal/14679671/homepage/call-for-papers/si-2025-000216
Guest Editors:
Steffen Knoblauch, Hao Li, Wenwen Li, Pedram Ghamisi, Benjamin Herfort, Alexander Zipf
Today is the 2026 International Day of Women and Girls in Science. My contribution is https://mastodon.social/@danstowell/116050729485377795
@Brett Buzzanga has joined the channel
@Bastiaan den Braber has joined the channel
@Rachel Walsh has joined the channel
@Mohamed Elabbas has joined the channel
@Michael Burke has joined the channel
@Taneea S Agrawaal has joined the channel
@Rodrigo Oyanedel has joined the channel
@McKalee Suzanne Steen has joined the channel
Hello everyone,
TECNIACÚSTICA® 2026 (57th Spanish Congress on Acoustics – XIV Iberian Congress on Acoustics) will be held in Granada, Spain, from 21–23 October 2026. Contributions are invited for the Bioacoustics structured session (A02-1).
While papers in English are welcome, the congress is primarily oriented toward the Spanish and Portuguese acoustics research community, including researchers based in Spain or Portugal as well as those working abroad. Researchers from other countries who are professionally connected to the Iberian acoustics community—for example through research, collaboration, or work in Spain or Portugal—are also very welcome to participate.
Submitted contributions will be included in the congress program and proceedings as Invited Communications (CI).
Submission details
• Abstract (maximum 250 words) and full paper must be submitted online following the conference guidelines.
• Instructions and formatting guidelines for abstracts and full papers are available here:
https://www.tecniacustica.es/TECNIACUSTICA2026/comunicaciones/normativa
• Online submission portal:
https://www.tecniacustica.es/TECNIACUSTICA2026/comunicaciones/envio-online
Important deadlines
• Title and abstract submission: 15 May 2026
• Notification of acceptance: 15 June 2026
• Full paper submission: 7 September 2026
• Early-bird registration deadline: 30 July 2026
• Congress dates: 21–23 October 2026
As in previous editions of TECNIACÚSTICA, members of the organizing institutions (SEA, SPA, EAA, FIA, ICA, UCA) benefit from a reduced registration fee.
The conference also offers awards and travel grants for young researchers, including:
• Travel Grants for Young Acousticians
• Andrés Lara Award
Further information about the congress can be found on the conference website:
https://www.tecniacustica.es/TECNIACUSTICA2026
If you are interested in contributing to the Bioacoustics session, please feel free to get in touch.
Best regards,
Ester Vidaña Vila and Marius Miron
Bioacoustics Session Chairs
@Francisco Gonzalez has joined the channel
@Amanda Swayze-Rebeaud has joined the channel
@Daniel Thornton has joined the channel
@Thongthornpatch Chamadol has joined the channel
@Salma Daadouch has joined the channel
@Lucas Mateo Waldburger has joined the channel
@Khushbu Sainani has joined the channel
@Edwin Arkel Rios has joined the channel
@Mahny Barazandehtar has joined the channel
Hello everyone,
I'd like to share with you the preprint of a tool paper I submitted recently. We have investigated how agentic AI could be used to support the debugging of machine learning projects. Happy to read your feedbacks!
https://arxiv.org/abs/2603.14099
Hi Sam,
I didn't know about that channel,thanks!
If you're going to CVPR and looking for accommodations, one of my PhD students is looking to share his room at a hotel one block away from the convention center (I think it's the Hyatt or Hilton, I don't remember). The room has two separate queen beds. Please DM me if you're interested and I can make introductions. (Hotel says they can do split invoices if that helps with reimbursement.)
Hi! I'll be presenting at the ML4RS workshop at ICLR2026 in Rio de Janeiro, Brazil. I'll be arriving in Rio on April 22 and was wondering if someone else will be there at that time and would like to explore Rio together before the conference starts? For example, it would be wonderful to visit Tijuca National Park! Please DM if you would like to coordinate 🤗
Hi Alice! I'd love to meet up, I'll send you a dm 🙂
Hi, if you have time I think you would love to visit our field station in the core of the Atlantic Forest, where we're using a lot of remove sensing and field measurements to guide conservation. See more at https://www.hu-b.org/
New IPBES Virtual Backgrounds Now Available!
Dear IPBES Members, observers and other interested stakeholders,
We are very pleased to share with you the news that a brand new collection of the very popular IPBES virtual backgrounds is now available and may be accessed here: https://ipbes.canto.de/b/M7I9I
With vibrant visuals ranging from marine species to endangered mammals and vivid fungi, everyone should find something appealing to brighten up their next video call (on any platform that allows you to customise your call background).
Please feel free to share these new designs widely with your networks and to please consider using them yourself.
With best regards,
Robert Spaull | Head of Communications
IPBES
Hey AI4C, I was wondering if any of you dabble in technical writing and would be open to chatting about it? Thank you 🙂
Out of curiosity, for those who do drone work for ecology, what drones do you use (especially for those in the USA, are there good alternatives to DJI)?
can you tell me about your needs? We have tested a dozen?
https://www.diu.mil/blue-uas-cleared-list
for a wading bird project in the everglades we used https://wisprsystems.com skyscout, but I think the truth is that is US manufactors are still getting up to speed.
we have a https://wingtra.com/?srsltid=AfmBOorcU_PW6Xo2DfXq9I1vzak_0Tp8yVVceRBIyH6q26LWUJesEbKd but don't seem to use it much.
We have worked a lot with Sensefly eBees and more recently, in the WildDrone project, on all kinds of DJI and custom made ones
Ah cool thank you so much for these!! I'm getting a lot of warnings on the blue-list website (ironic?), but yes, I am interested in the "easiest to use" drones for basic surveys. I know DJIs are pretty solid but seem no longer viable in the US? Especially interested in those using them for coastal/shallow water ecosystem mapping (slightly below surface) tasks (bonus points if anybody uses them for water samples).
Has anyone used one of the Remarkable Pro 2 tablets (with color eInk) for reading papers? I am just curious how functional they actually are.
Specifically that model? I use the OG remarkable, if that's helpful at all. Long story short, my recommendation has always been “if you're looking for the closest possible thing to paper, but digital, and that's all you want, the remarkable is good. Else get an iPad”
The new one has color, which should be nice for figures. But I wasn't sure if the color eink would reduce loading/scrolling speed. I am just looking for a reading device with easy transfer of files from desktop to tablet.
I also have a reMarkable 2, with a greyscale display. AFAIK, the colour rendering on the Pro is limited to something like 8–12 colours, but if your need is only to read simple figures, that might do the trick. To be honest, I rarely miss seeing colours while reading e.g. articles or reports as PDF, and I cherish the chance to read from a non-backlit display to give my eyes a break. E-paper will not render as quickly as a regular tablet though. Maybe you can find a video online that shows it in use, to get an idea of the latency?
The Allen Institute is looking for input on this survey about AI use in science: https://docs.google.com/forms/d/e/1FAIpQLSfjh-iv3yAxVimruftEEq8gxL3ga_eNKWriFwT6bd810Idwlw/viewform
Hi everyone! We've organized a GeoAI meetup for CVPR Denver next week on Friday, June 5th from 5-7PM at Woods Boss Brewing! All are welcome -- GeoAI is broadly defined and we encourage anyone working on related topics to join too :)
Hi all - we are just starting to look at plant species detection/semantic segmentation from 360 degree camera imagery (GoPro MAX2). Does anyone know of any libraries/repos related to this? Thank you!
Oh good question. I'm not aware of any of the models that are trained to handle the distortion from the 360 flattening, so unless you have a dataset large enough to train your own model, the best next strategy would probably be to flatten each frame, compensate for distortion and then run the image through the standard models.
For example, I used this library once for a similar conversion: https://github.com/asteran32/Spherical-to-CubeMap-Image-Conversion
Thanks Keenan - yes that was going to be our first approach. Thanks for the link to that library, will take a look. 👍
great. Yeah I wish there was such a model trained on 360 cameras that I could recommend, but I haven't heard or seen one.
Hi Hugh, I'm working on a project to do this for under canopy forest settings! It would be cool to compare notes. We use a similar cube-mapping approach, a semantic model we trained, and a tool I've led called geograypher to project the results into 3D.
Hi David - thanks for connecting. Would be great to chat and hear about how you’ve approached this. We are collaborating with some researchers at the University of Galway (Ireland). Let me circle back with them and find some times we could meet. Thanks again!
This is awesome (my personal opinion as a non-Brit)... https://www.bbc.com/news/articles/cvgzrpe6mzjo
Hi everyone,
We are excited to announce that we have just released the FishNet Challenge under the 3rd CV4Ecology Workshop @ ECCV 2026.
The challenge focuses on text-guided open-set fine-grained fish classification. More details can be found in the poster below and on our website:
https://cv4e-workshop.github.io/competition.html
You are very welcome to check it out and participate!
Mark your calendars for the upcoming AI + Environment Summit that will take place on September 30th at the Innovation Park Zürich. This event will be an exciting day filled with talks from inspiring speakers (from academia, industry, and non-profits), thematic sessions, and a poster session. Breakfast, lunch, and an apero will be provided, along with sustainably-sourced coffee. If you want to feature your work, you can submit a poster to be presented. Learn more at https://ai-environment-summit.com/ and register here.
Hi community, are there any bird watchers here?
I am currently in the Indian Himalayas, and there is a lovely bird that wakes me up every morning with its sweet song/call, but I am having a hard time identifying it. I have the audio recording and I have tried BirdNet, but its suggestions aren't matching with my recording. Would be great if someone could help me with this, attaching the audio file.
Thanks already!
@Swayam Thakkar Perhaps you or someone you know might be able to ID it?
I don’t know Indian birders, but posting on iNaturalist is usually the fastest way to find an expert to ID
My bet is on a white browed fantail. But I am not 100 percent sure. Ive sent it to a senior for confirmation :)
This was a nice hit of nostalgia from my childhood in Bombay, so thanks for the recording and the ID 🙂
I knew I'd heard something very similar before, but turns out there's a lot of fantails that sound and look similar to the untrained ear
Yes, could be one of the fantails. And also perhaps a verditer flycatcher is singing in the background.
Speaking of bird calls: https://www.theguardian.com/science/2026/jun/26/human-animal-communication-step-closer-scientist-wins-prize-for-decoding-birdsong
Dear everyone, our fishnet challenge @ ECCV26 CV4Ecology workshop is still ongoing, you are very welcome to have a look and participate at the competition: https://cv4e-workshop.github.io/competition.html 🙌🙌
I have been having to do lots of embedded work for different ROVs recently which although fun has lots of problems, so I decided to try making a little game version with most of the fun and no practical engineering problems. This is right now mostly a proof of concept, but I hope to keep improving it / making it more fun here and there as I have time. Check it out if you would like: bloop.quest
Hi everyone! 👋 Looking for advice from anyone who's done long-term outdoor/wildlife sensor deployments.
We're planning a 2–4 week unattended field deployment for multi-modal data collection. Most of the time it's low-power sensing, but we periodically wake a Raspberry Pi 5 for heavier compute — so power is our main bottleneck. Depending on the wakeup schedule, the node draws roughly 0.1–1 W average.
Three power options we're weighing:
- Voltaic V75 (72 Wh) + solar panel. Purpose-built for field sensing — always-on output, native solar input, popular with camera-trap folks. Worries: 72 Wh alone won't carry 2–4 weeks, so we'd depend heavily on solar (cloudy stretches scare us), and its 5V/3A max output + high-temp cutoff might not love a Pi 5 in summer.
- 12V LiFePO4 (~20–40 Ah) + buck converter (12V→5V) + solar with MPPT. Much bigger capacity, stable Pi power, and LiFePO4 handles heat well. Downside: heavier, bulkier, needs a proper enclosure.
- Battery-only, duty-cycled — like Conservation X Labs' Sentinel (conservation-x-labs-1.gitbook.io/sentinel-docs-english/getting-started/quick-start-guide). It wakes on a set cycle time / trigger count instead of every trigger, and per their docs gets ~200 days on 7 batteries at a 6-hour cycle. Our compute load is heavier than a trail-cam add-on, so we'd expect much shorter life — but the wake-on-cycle model fits how we already schedule the Pi. One practical downside: it runs on a tray of AA cells, so turnaround between deployments means either buying fresh lithium primaries every time or pulling out rechargeables and cycling them through a charger a few at a time — feels fiddly compared to just plugging in one pack.
Questions for anyone who's been through this:
What battery + solar sizing actually held up for a multi-week SBC deployment?
Any real-world gotchas with buck converters, MPPT/PWM controllers, or self-discharge on all-in-one power banks?
Has anyone tried something like option 3 (battery-only, duty-cycled) with an SBC — how far did it realistically get you?
For those running AA-powered nodes: do you just bulk-buy lithium primaries, or is there a sane rechargeable-AA workflow that doesn't drive you crazy?
Any parts lists or deployment write-ups would be hugely appreciated. Thanks! 🙏
Hi @Yi Ding, SBCs are generally not practical for outdoor deployment but precisely timed ON/OFF schedules can help you prolong your battery life.
I'm sharing a sample SOLAR energy and Battery charging balancer spreadsheet.
Dear everyone, a friendly reminder about the CV4Ecology @ ECCV 2026 Workshop:
The archival track closes on July 15 (AoE). If you are interested in the workshop and have not submitted your paper yet, please remember to do so before the deadline.
The non-archival track will remain open until August 14 (AoE)
🌿 Workshop date: September 8, 2026
🔗 cv4e-workshop.github.io
Thanks for your attention ^-^
For our airborne marine biodiversity surveys we have some overland imagery as the planes take off. The model keeps finding sleeping dogs and calling them seals. This is like the 5th time.
For anyone wondering, this is what a seal looks like in the imagery.
@Ben Weinstein thanks for sharing this just made my day 😂
Sheepdog? Lol who is good at fine grained dog breed classification 😂😂
From the ecological society of America meeting:
A friend/colleague sent me this earlier, that I think is also relevant: octogroup.org/programs/ai-snapshot-report-2026
Clearly there are many genuine issues to confront, as well as a lot of fear/misunderstanding about AI. @Justin Kitzes on this Slack is up for election for the ESA governing board this year; maybe he can help 😉
Ha well we'll see if the members are interested in electing an AI guy 😄
📢 Reviewer Recruitment — CV4Ecology @ ECCV 2026
We are recruiting reviewers for the 3rd Workshop on Computer Vision for Ecology (CV4Ecology), taking place at ECCV 2026 on September 8 in Malmö, Sweden.
We welcome researchers working in computer vision, machine learning, ecology, biodiversity, conservation, and related areas to join our reviewer pool and help support high-quality interdisciplinary research.
📝 Reviewer sign-up form:
forms.gle/…
🌐 Workshop website:
cv4e-workshop.github.io
Please feel free to share this with colleagues and students who may be interested. Thank you for helping us spread the word!
Feel free to share on the TorchGeo slack as well
All Watched Over by Machines of Loving Grace" is a famous 1967 counterculture poem by Richard Brautigan. It paints a picture of a peaceful future in which technology and nature live together in harmony. Sigh.
All Watched Over by Machines of Loving Grace
Richard Brautigan, 1967
I like to think (and the sooner the better!)
of a cybernetic meadow
where mammals and computers
live together in mutually
programming harmony
like pure water
touching clear sky
I like to think
(right now, please!)
of a cybernetic forest
filled with pines and electronics
where deer stroll peacefully
past computers
as if they were flowers
with spinning blossoms.
I like to think
(it has to be!)
of a cybernetic ecology
where we are free of our labors
and joined back to nature,
returned to our mammal
brothers and sisters,
and all watched over
by machines of loving grace
@Michael Bunsen has a nice artwork of this
Adam Curtis also has a documentary titled this which looks at the dark side of how counter culture sentiments like these have been co opted by corporate interests :/
(Has a fantastic soundtrack)
https://youtu.be/I6EBpLfLHCA?is=Nxapx9rUf-2uKHsq
Also one of the reasons I am attracted to this kind of work we do, yet very concerned with how dangerous our seemingly benign work can be.
Thanks for sharing this @Tanya Berger-Wolf! The last paragraph is particularly lovely.
Humans have an unfortunate ability to turn every utopia into a dystopia…
@Tanya Berger-Wolf indeed. you might enjoy the first season of Nice Try! season 1 details the many ways humans have tried and failed to create utopia. the Biosphere episode is a cautionary tale! open.spotify.com/episode/…?si=…
Hey! I built this for a job application that didn't work out; but it's genuinely fun to play with, and it felt like a shame to leave it sitting in a folder, so I thought of sharing it with this community.
It's a platform for exploring geospatial data by just asking in natural language. You type a question in plain English, and it works out which data it needs, queries it, and hands back a map or a chart. This one's loaded with a subset of bird data from eBird Status and Trends. 🐦 -- you can access it here: huggingface.co/spaces/GerardCB/perch
My favourite thing to type into it: "Animate how Ruby-throated Hummingbird abundance shifts through the year". You can also click any bird's photo to hear its call, which I enjoyed building far more than I should have. Enjoy!
Hi everyone,
We are conducting a research study on the use of AI, particularly large language models, to generate descriptions of biological species, and we are looking for participants to help us evaluate the quality of these AI-generated descriptions.
No expertise in AI or biological species is required. We are particularly interested in intuitive evaluations from people with different backgrounds. The survey takes approximately 10 minutes and is available in English, German, Japanese, and Chinese.
If you have a few minutes, we would greatly appreciate your participation and contribution to our research:
sites.google.com/view/biox-kics-zert/home
Please also feel free to share the survey with colleagues or students who may be interested.
Thank you very much for your support.
Best regards,
Yutong Zhou
Postdoc, Leibniz-Centre for Agricultural Landscape Research (ZALF)
Hey everyone,
to work with our own camera trap data we have developed a lot of little functions and some tools that help automate our processing of them and deal with some issues. We were playing around with the idea to combine them into a python package for people to embed the functions into their processing pipeline. I wanted to check with the community if you feel there is a use for it. We would also generally be interested to make it open source so it can grow over time when other people add to it.
Functionalities we are thinking on adding:
general functions
- automatic extraction of temperature from the CT image pixels (regardless of camera brand)
- automatic extraction of date time from the CT image pixels (regardless of camera brand)
- extraction of the camera trap brand from the CT image, and if different CTs are the same model of camera
- categorizing each image into a day or night image. This is based on colored vs. grey scale images but also works if all images are grey. We are thinking about adding functionality to do this by temperature as well so it would work for small mammal boxes
- finding "view changes": something interferes with the camera trap so the view that it captures after is different. Eg. it gets knocked over and looks at the sky from that point on.
- finding "time shifts": sometimes camera traps put the wrong time stamps (often due to a reset), this tool finds the correct ones by using day/night patterns
some functions based on MegaDetector output + AI training input processing
- detecting "stationary boxes": e.g. times were MD keeps identifying a specific rock as an animal. This function identifies these boxes and filters them out depending on a custom set threshold (time or/and how often the box appears)
- cropping images using MD output , but all crops are made into squares (with padding if wanted) that are better suited for NN input
- data splitting by location or different camera trap runs to keep training / testing / validation set unbiased based on a simple greedy algorithm
We would be grateful to hear your thoughts on the need / usefulness of any of these functions or if you can think of any that you would have a need for.
thank you for your help,
Aimée
I have found myself implementing bad one-off versions of all of these countless times, so I think these would be extremely useful if packaged in a re-usable way! I think you might be able to get some metadata from EXIF tags rather than pixels at least for some manufacturers?
Everyone's needs are a little different, so, YMMV, but FWIW I have versions of most of these in the MegaDetector Python package. In no particular order...
Re: data splitting into train/val by locations...
This function does exactly that, with a best effort to preserve class distribution:
https://megadetector.readthedocs.io/en/latest/utils.html#module-megadetector.utils.split_locations_into_train_val
Re: cropping and resizing...
Cropping based on MegaDetector results, with preservation of folder/filename structure, with a best effort to preserve EXIF data:
https://megadetector.readthedocs.io/en/latest/postprocessing.html#module-megadetector.postprocessing.create_crop_folder
Bulk resize of a folder of images (e.g. to square), also with a best effort to preserve EXIF data:
https://megadetector.readthedocs.io/en/latest/visualization.html#megadetector.visualization.visualization_utils.resize_image_folder
So if you want to crop to square, I would use createcropfolder() then resizeimagefolder().
Re: day/night classification and finding time shifts...
I'm assuming you want to handle situations where the clock was untrustworthy, otherwise you would just use the clock. I have occasionally solved this problem by detecting grayscale pixels, as you suggested, using:
https://megadetector.readthedocs.io/en/latest/visualization.html#megadetector.visualization.visualization_utils.gray_scale_fraction
This is camera-agnostic, so that's good, although it depends on the camera producing grayscale or near-grayscale output at night, which is not universal; some projects use a white flash at night. E.g. Snapshot Serengeti uses a white flash at night. If you don't want to depend on grayscale output, a reliable way to do this is to use EXIF information about whether the flash was triggered (the "Flash" EXIF field, when present), or what the exposure time was (typically using the "ExposureTime" field). This quite reliable; exposure time is quite bimodal depending on whether the flash fired.
I recently worked with the Snapshot Serengeti team to look for incorrectly-set clocks over a large pile of historical data, and this approach worked really well (confirmed against a handful of cases where they had manually discovered clock issues). More information here:
https://github.com/agentmorris/camera-trap-day-night-classifier
I didn't really wrap this up into a proper function, but basically by comparing EXIF information about flash/exposure to the clock time, it was easy to detect cameras that were way off. It's almost impossible to detect cameras that are like 20 minutes off, but fortunately that kind of error is probably not ecologically significant.
Re: stationary boxes...
I would use some caution here, because sometimes animals just go to sleep in front of the camera for a long time, and more importantly, sometimes people/animals consistently enter the frame at exactly the same place because of the alignment of the camera relative to a path. In both of these cases, you may see the same box repeated in exactly the same place many times and find that it's not a rock. This is especially true when cameras are facing narrow trails where people and animals enter in a very predictable way.
However, there's good news! You can still use this repetition to speed review up by like a zillion percent, but I treat this as a semi-automated process where I review one exemplar of each repeated detection to quickly confirm it's a rock. I summarize the process I use here:
https://lila.science/repeat-detection-elimination
I never run a MD/SpeciesNet batch without doing this process. It's a bit of an art, but once you get used to it, it's very fast. I typically spend around five minutes of human time per million images in a batch, which means reviewing maybe 1000 exemplars, but most of the time you're just banging the page-down key very quickly.
The core function that finds repeat detections is here:
https://megadetector.readthedocs.io/en/latest/postprocessing.repeat_detection_elimination.html#find_repeat_detections---CLI-interface
The code is not magical, but there are a few things I would take away from it:
- Rocks don't move, but cameras move, and also other false-trigger objects move (e.g. branches), so you need to allow some looseness in your definition of whether a box is repeating in exactly the same location. One of the parameters I use for this process is an IoU threshold.
- Other useful parameters: occurrence threshold (a detection has to appear at least N times to be considered a candidate suspicious detection (i.e., a candidate rock), typically N=20 within a camera-year or so), max area (this process doesn't work well for detections the size of the whole image, because those are often animals very close to the camera)
- Comparing a million boxes to a million boxes (to look for repeats) can be slow; I have been super-duper happy with the fastquadtree package for making this fast (https://github.com/Elan456/fastquadtree)
Re: camera trap brand/time/etc....
Time is available in EXIF data 100% of the time, the only reason you would want to pull it off of the camera is if you are working with historical data where the images have lost their metadata.
In my experience, as Timm suggested, the manufacturer name is also almost always available in metadata (I don't think I've seen a case where this isn't true, though I would guess those cases exist). Consequently, I would not use pixels to determine manufacturer name.
Temperature, on the other hand, is very unpredictable in terms of being available in metadata. Also moon phase, that's a fun one.
FWIW I have a function for bulk, multi-threaded extraction of EXIF metadata from a folder of images here:
https://megadetector.readthedocs.io/en/latest/data_management.html#module-megadetector.data_management.read_exif
And despite my caveats about not trying to do this from pixels, I have some rather experimental code that tries to pull all of these things directly from pixels here:
https://megadetector.readthedocs.io/en/latest/data_management.html#module-megadetector.data_management.ocr_tools
This uses the pytesseract package and a bunch of heuristics and cascades of regular expressions.
I would not do this unless you are really sure you can't depend on EXIF, and in that case I would think of it as a slightly interactive process that you need to spend a few minutes tuning for a dataset.
Also if you need to do this at relatively low volumes, I would just use a VLM (frontier VLMs are very very very good at this task, I wrote the ocr_tools module before VLMs worked).
All everything I just said applies to images; if you are using video, all bets are off in terms of what's available in metadata. Basically nothing is available in video metadata, and your best best is to try to keep videos tied to images that were collected at the same time.
Re: view changes...
I have not looked at this, just mentioning it here for completeness.
Also FYI if you want to use any of the functions I mentioned without installing the heavier dependencies from the megadetector Python package (e.g. all the Torch stuff required to run MD), you can install megadetector-utils instead (pip install megadetector-utils). "megadetector" and "megadetector-utils" are identical other than the latter not having the Torch dependencies.
A few months ago I built WildTrack (github.com/GerardCB/wildtrack) for wildlife detection, segmentation, and tracking in video data, which might cover some or extend the MegaDetector output + AI pipeline functionalities you listed. Its functionalities have now been partially covered by models like SAM3, but WildTrack is built to be modular and you can swap in your own detector or classification models.
I generally agree with Dan’s points here.
For EXIF, camera brand, and timestamp handling, we have ended up building pretty much the same kinds of utilities ourselves (for the specific camera brands we maintain). For video metadata, we create XMP sidecars and then batch-write the relevant metadata there to keep the data consistent.
I do think view-change detection would be a useful addition, though. We have at least two common cases where this matters:
- Some of our cameras are maintained by volunteers or local community members, and occasionally a camera is manually moved to a new position without that change being properly recorded.
- Cameras can also be knocked over by animals. In some cases this affects the camera working days, so we currently depend mostly on people noticing and marking these changes during manual data review.
For stationary boxes / repeat detections, I have been a long-time user of the MegaDetector RDE scripts and have found them very useful. I previously made some quick modifications to support video-related workflows as well (I haven't cleaned those changes up enough to contribute upstream).
More recently, I have been working on a WASM client specifically for RDE (to address dmorris.net/task-viewer?file=https%3A%2F%2Fraw.gi…&item=client-side-rde-tool). I think RDE can probably work as a independent tool: it is largely orthogonal to the rest of the camera-trap processing workflow.
@Dan Morris I'll let you know when it is a bit more complete (also, I’m still looking for a good name for the tool), and we may discuss how it could integrate with the existing GUI workflow such as AddaxAI and biowatch (or whether they will have some thing similar themselves). cc @Peter van Lunteren @Arthur Caillau
Thank you very much for your in depth feedback.
I see that for some cases like cropping, data splitting and finding repeat detections great functions already exist and it would not make sense for us to "reinvent the wheel".
For context, we work with long time monitoring data (since 2012) where we have quite some "veteran" camera traps that have many issues. While we generally try to pull everything we can from the exif data we did have some cases where metadata was lost and we pulled it from the image. We had special difficulty to get the camera trap brand from exif data. So we thought some people might face these issues as well but maybe its not as common as we have thought. These are just small helper functions but maybe useful if someone does not program a lot?
For time shifts:
@Dan Morris yes I was referring to cases where the recorded time stamps are not reliable. Around 8-10% of our camera traps have time issues with the clock resetting. Our tool is sensitive down to a minute for a shift. It additionally also finds the time delta it was shifted by and returns the time stamps the images should have. We are still running simulations to see how close we get to the real time stamp with that. We have also been informed of cases from other projects where there is a clock drift, so where the minute of the CT is slightly longer or shorter than a standard minute and time shifts happen gradually. Our tool would also deal with that as they are really hard to spot and manually frustrating to correct. The input it takes is the order the images were taken + if an image is day or night (however you wish to classify that).
For view shift detections:
@Yunxuan Chai (wsyxbcl) we also have those 2 cases, which was our motivation to find a solution as it matters for analysis. The knocking over by animals also often coincides with a time reset as the camera trap resets from the hit it takes. It is not quite tested to our satisfaction but an initial semi-robust version stands.
Again thank you everyone for the extensive feedback and linking so much information. It was very helpful and we can now better judge how to focus our efforts.
Hi, i've been working semi-part time in two things that are of interest to us: first one is extracting object distance from camera traps. This is more of an engineering problem really - i've already made small app to do the depth estimation and let user create a calibration. This is important because of the "Camera trap distance sampling" methods for animal abundance are extremely time consuming with the user having to check the calibration video alongside the animal detection video to estimate the distance at which the animal is. For this - a module to co-register two camera trap images would be great. I'm sure it can be explored with SIFT algorithms but i haven't had the time yet. And then tying this to correct the depth estimation - here is a working depth estimator: github.com/nunosa-wwf/CameraTrap_DistanceEstimator
Yunxuan wrote:
...also, I’m still looking for a good name for the tool
DeRocker, DeRock, Unrepeater, RockRemover, RocksBeGone, UnRocker, RepeatRemover, RepeatsBeGone
Among those, I think I vote for "RocksBeGone".
Hi all, reading the list I am wondering if anyone knows of a tool to extract information from the image on whether it is time-lapse/field-scan triggered or motion-triggered? I have been working both with Agouti and AddaxAI, neither of which extracts this info (for my camera trap models at least, I know Agouti does that for Reconyx..). Thanks in advance!
a tool to extract information from the image on whether it is time-lapse/field-scan triggered or motion-triggered?This is quite dependent on the specific camera model. Two possible approaches:
- Metadata: I would take a few test images from both time-lapse/field-scan and PIR-triggered (which I believe your motion-triggered means) modes, inspect the full metadata with ExifTool or Exiv2, and see whether any tags or values differ consistently between the two modes.
- Image overlay/watermark: For cameras I have worked with, the two trigger modes produce different information in the bottom-bar overlay/watermark. In that case, you could crop the relevant region and use OCR (or potentially a VLM) to classify the trigger mode from the displayed text/symbols.
Let's first see if either of these applies.
will try that, thanks! Point 2 is exactly what I had in mind, and wondering if others had done so already.
The temp one! I'll dig into what Dan mentioned
More recently, I have been working on a WASM client specifically for RDE (to address dmorris.net/task-viewer?file=https%3A%2F%2Fraw.gi…&item=client-side-rde-tool).A follow-up on this: For anyone interested in RDE (Repeat Detection Elimination in MegaDetector post-processing) , I vibed up a tool called RocksBeGone — thanks @Dan Morris for the name.
You can try it directly at rocksbegone.camtra.pw. The site only serves the app; clustering, image/video decoding, review, and export all run locally in your browser via WASM. MegaDetector JSON, media, and metadata are never uploaded.
If your working machine or data-processing environment is completely offline, you can also download the release here:
github.com/wsyxbcl/RocksBeGone/releases
Would love any suggestions, bug reports, weird camera/media cases, or thoughts on the workflow!
@Yunxuan Chai (wsyxbcl) amazing! Looks great, I will use this in lieu of my normal RDE process the next time I'm processing a batch of images, and report back with any issues I hit. Thanks for sharing!